Tutorial - ?· Jtest Tutorial 1 Tutorial TutorialJtest Tutorial Welcome to the Jtest Tutorial. ... ParaSoft’s…

  • Published on
    19-Aug-2018

  • View
    220

  • Download
    1

Embed Size (px)

Transcript

  • Jtest Tutorial

    1

    Tu

    torial

    TutorialJtest TutorialWelcome to the Jtest Tutorial. This tutorial walks you through how to per-form common Jtest tasks using example files.

    Please note that although the four types of tests (static analysis, white-box testing, black-box testing, and regression testing) are dis-cussed separately, Jtest can perform all of these tests with just one click of the Start button.

    This tutorial contains the following lessons:

    Lesson 1 - Performing White-Box Testing on page 2

    Lesson 2 - Performing Static Analysis on page 13

    Lesson 3 - Performing Black-Box Testing on page 23

    Lesson 4 - Performing Regression Testing on page 51

    Lesson 5 - Testing a Set of Classes on page 54

    Lesson 6 - Using User-Defined Stubs on page 59

    Lesson 7- Using JUnit Test Classes with Jtest on page 69

    Lesson 8- Using Jtest in a Multi-User Environment on page 79

  • Lesson 1 - Performing White-Box Testing

    2

    Tu

    tori

    al

    Lesson 1 - Performing White-Box TestingWhite-box testing checks that the class is structurally sound. It doesnt test that the class behaves according to the specification, but instead ensures that the class doesnt crash and that it behaves correctly when passed unexpected input.

    White-box testing involves looking at the class code and trying to find out if there are any possible class usages that will make the class crash (in Java this is equivalent to throwing an uncaught runtime exception).

    Jtest completely automates white-box testing by automatically generating and executing test cases designed to fully test the class. During the test, Jtest executes the class using a symbolic virtual machine, then searches for uncaught runtime exceptions. For each uncaught runtime exception that is detected, Jtest reports an error and provides the stack trace and the calling sequence that led to the problem.

    This lesson contains the following sections:

    Performing White-Box Testing: Overview on page 2

    Performing White-Box Testing: Example on page 3

    Viewing Automatically Generated Test Cases on page 6

    Suppressing Exceptions on page 7

    Viewing a Text or HTML Format Report on page 7

    Setting an Object to a Certain State on page 8

    Performing White-Box Testing: OverviewTo perform white-box testing, just tell Jtest what class or set of classes to test then click the Start button.

    If you test a project, results will be displayed in the Class Name> Errors Found> Uncaught Runtime Exceptions branch of the Project Testing UI's Results panel.

  • Lesson 1 - Performing White-Box Testing

    Tu

    torial

    If you test a single class, results will be displayed in the Uncaught Runt-ime Exceptions branch of the Class Testing UIs Errors Found panel.

    Performing White-Box Testing: Example

    1. Go to Jtests Class Testing UI. (This UI opens by default when you launch Jtest).

    2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.

    3. Browse to Simple.class (in /examples/eval) using the Browse button in the Class Name panel.

    4. Click the Start button in the tool bar.

    Jtest will perform static and dynamic analysis on the class. A dialog box will open to notify you when testing is complete. Information on test progress will be displayed in the Test Progress panel. Problems found will be reported in the Errors Found panel.

    3

  • Lesson 1 - Performing White-Box Testing

    4

    Tu

    tori

    al

    You will see two types of errors reported in the Errors Found panel. For this example, were going to focus on the uncaught runtime exception found.

    The Errors Found panel will list the following uncaught runtime exception:

    This error message reveals that there is some input for which the class will throw an uncaught runtime exception at runtime. This could cause the application running this class to crash.

    To see a stack trace like the one the Java virtual machine would give if this uncaught runtime exception were thrown, expand this branch.

  • Lesson 1 - Performing White-Box Testing

    Tu

    torial

    To see an example usage of this class that would lead to the reported uncaught runtime exception, expand the Test Case Input branch.

    The error shows us that the "startsWith" method is implemented incor-rectly. The method should return false for the argument "" and "0" instead of throwing a runtime exception.

    If the error is not fixed, any application using this class will eventually crash or give incorrect results.

    To view the source code of the class (with the problematic line of the stack trace highlighted), double-click the node containing the exceptions file/line information.

    5

  • Lesson 1 - Performing White-Box Testing

    6

    Tu

    tori

    al

    Viewing Automatically Generated Test CasesTo see a selection of the test cases that Jtest automatically generated for this test, click the View button in the tool bar. The View Test Cases win-dow will open.

    In the View Test Cases window, expand the Automatic Test Cases branch. This will display a list of this classs methods. Click any method with a number greater than zero by it, and open the Test Cases branch, or press Control and click your right mouse button, then choose Expand Children from the shortcut menu that opens. The inputs that Jtest gener-ated for each method are now displayed.

  • Lesson 1 - Performing White-Box Testing

    Tu

    torial

    Suppressing ExceptionsYou can tell Jtest to suppress uncaught runtime exceptions that you do not expect to occur or that you are not concerned with by adding Design by Contract tags to your code. To have Jtest suppress errors for inputs that you do not expect to occur, use the @pre tag to specify what inputs are permissible. To have Jtest suppress expected exceptions, use the @exception tag to specify what exceptions you want Jtest to ignore.

    For example, if you wanted to suppress reports of an expected Negative-ArraySizeException that occurs when a negative index is used as an index to an array, you might enter the following comment above the appropriate method:

    /** @exception java.lang.NegativeArraySizeException */

    This not only tells Jtest to ignore the exception, but it also makes code easier to understand and maintain.

    If you use the @pre tag to indicate valid method inputs, and then use ParaSofts Jcontract to check Design by Contract contracts at runtime, you will automatically be alerted to instances where the system passes this method any unexpected inputs.

    Viewing a Text or HTML Format ReportJtest automatically creates a report for each test. You can view this report by clicking the Report button. By default, this report is formatted in text (ASCII) format. If you would like Jtest to generate HTML reports (e.g., if you want to post the report on your development intranet), choose Prefer-

    7

  • Lesson 1 - Performing White-Box Testing

    8

    Tu

    tori

    al

    ences> Configuration Options> Report Format> HTML before you click the Report button..

    By default, the report file will be saved in /u/user-name/results. To prompt Jtest to save reports to a different location, mod-ify the Common Parameters> Directories> Results parameter at the Global, Class, or Project level.

    Setting an Object to a Certain StateIn some cases, you may want to set up an initial state prior to testing a class. For example, suppose that a class is used as a global object con-

  • Lesson 1 - Performing White-Box Testing

    Tu

    torial

    taining static member variables accessible by any other project within the application. When Jtest tests an object that uses this static member vari-able, a NullPointerException will result because the variable has not been set. This problem can be solved by giving Jtest initialization code.

    For example, the file ExecGlobal.java (in /lesson1/ExecGlobal.java) contains a static variable that is assigned by its ancestor Exec, which is defined in the Exec.java file (also in /lesson1/Exec.java). In this example, the instantiation on the Exec object is performed by the Exec objects "main" method, which is defined in the Exec.java file. When Jtest tests the ExecTest objects doTest method, it does not know that a certain state is assumed by the doTest method. Thus, initialization needs to be per-formed.

    To see why initialization is necessary:

    1. Go to Jtests Class Testing UI. (This UI opens by default when you launch Jtest).

    2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.

    3. Use the Browse button in the Class Name panel to choose ExecTest.class (in /lesson1/ExecTest.class)

    4. Test the class by clicking the Start button in the Class Testing UI tool bar.

    A NullPointerException is reported because the assumed state is not available. You will need to perform static initialization on the ExecTest class to ensure the objects referenced by this class are in the assumed

    9

    ../lesson1/ExecGlobal.java../lesson1/Exec.java../lesson1/ExecTest.class

  • Lesson 1 - Performing White-Box Testing

    10

    Tu

    tori

    al

    state.

    To perform the necessary initialization:

    1. Click the Class button to open the Class Test Parameters win-dow.

    2. In the Class Test Parameters window, open Dynamic Analysis> Test Case Generation> Common.

    3. Double-click the Static Class Initialization node.

  • Lesson 1 - Performing White-Box Testing

    Tu

    torial

    The Static Class Initialization window will open.

    4. Enter the following initialization code in the Static Class Initializa-tion window.

    new Exec ();This will create an instance of the "Exec" object before the class "ExecTest" is tested.

    11

  • Lesson 1 - Performing White-Box Testing

    12

    Tu

    tori

    al

    5. (Optional) If you want to check the method, choose Save & Check from the Options menu.

    6. Choose Options> Save.

    7. Choose Options> Quit.

    8. Click Start in the Class Testing UI tool bar to retest the class. You no longer receive a NullPointerException because an instance of Exec is available when testing the ExecTest object.

  • Lesson 2 - Performing Static Analysis

    Tu

    torial

    Lesson 2 - Performing Static AnalysisDuring static analysis analyzes source code to expose violations of cod-ing standards, language specific rules that help you prevent errors.

    Jtest enforces the following types of coding standards:

    Traditional coding standards: Traditional coding standards are rules that apply to constructs within the class under test. A tradi-tional coding standard might test whether or not a files source code contains a construct that has a high probability of resulting in an error. For example, one traditional coding standard checks that you use equals instead of == when comparing strings (writing == when you should have used equals causes the pro-gram to check if two strings are identical, rather than check if two strings have the same value).

    Global coding standards: Global coding standards are rules that ensure that projects use fields, methods, and classes wisely. A global coding standard might check that a project does not con-tain logical flaws and unclear code such as unused or overly-accessible fields, methods, and classes; such problems increase the probability of an error being introduced into the code, and may also make the code less object-oriented.

    Metrics: Metrics are measures of the size and complexity of code. When Jtest performs static analysis, it also measures your classs and (if applicable) projects metrics; it reports all metrics in the Metrics window, and reports a static analysis violation for any class metric that is outside of the bounds that have been set for that metric.

    Custom coding standards: Custom coding standards are rules specially tailored to your own or your groups unique coding style. You can create such customized rules using Jtests RuleWizard feature.

    Static analysis is performed automatically when you test a class or project.

    13

  • Lesson 2 - Performing Static Analysis

    14

    Tu

    tori

    al

    When enforcing all static analysis rules except for global static analysis rules, Jtest statically analyzes the classes by parsing the .java source and applying a set of rules to it, then alerting you to any rule violations found.

    When enforcing global static analysis rules, Jtest scans all of the projects .class files to collect global usage information, uses this information to check if each class violates any rules, then alerts you to any violations in the classes under test. Because Jtest uses .class files (rather than .java files) to enforce global static analysis rules, you can perform global static analysis even when .java files are not available.

    Note: Global static analysis can only be performed while testing a project.

    This lesson contains the following sections:

    Performing Static Analysis: Overview on page 14

    Performing Static Analysis: Example on page 14

    Viewing Metrics on page 17

    Customizing Static Analysis on page 18

    Performing Static Analysis: OverviewTo perform static analysis, just Jtest what class or set of classes to test then click the Start button.

    If you test a project, results will be displayed in the Class Name> Errors Found> Static Analysis branch of the Project Testing UI's Results panel.

    If you test a single class, results will be displayed in the Static Analysis branch of the Class Testing UIs Errors Found panel.

    Important: Because Jtests static analysis tests at the source level, Jtest will only perform static analysis on classes whose .java files are available.

    Performing Static Analysis: Example1. Go to Jtests Class Testing UI. (This UI opens by default when

    you launch Jtest).

  • Lesson 2 - Performing Static Analysis

    Tu

    torial

    2. If a class is already loaded into the Class Testing UI (i.e., if you see a class name in the Class Name field), click the New button to clear the previous test.

    3. Browse to Simple.class (in /examples/eval) using the Browse button in the Class Name panel.

    4. Click the Start button in the tool bar.

    Jtest will perform static and dynamic analysis on the class.

    Information on test progress will be displayed in the Test Progress panel. Errors found will be reported in the Errors Found panel.

    You will see two types of errors reported in the Errors Found panel.

    For this example, were going to focus on the static analysis violations.

    The Static Analysis branch of the Errors Found panel should list the fol-lowing violations:

    15

  • Lesson 2 - Performing Static Analysis

    16

    Tu

    tori

    al

    To see more information about a violation, expand the violations branch. For example, expand the violation of the PB.TLS rule.

    This violation message reveals that the developer inadvertently wrote "case10" instead of "case 10". If the class is not fixed, it will give incorrect results when it is passed the value 10.

    To learn more about any rule that Jtest enforces, right-click the topmost branch of the error violation that you would like explained, then choose View Rule Description from the shortcut menu.

    A dialog box that contains a description and example of the selected rule violation, as well as a suggested repair, will open.

    You can also view the source code responsible for the violation; to do t...

Recommended

View more >