Unit Testing

Definition

Unit testing is the testing of an individual unit of software, typically by its developer or by a peer programmer.

Objectives

The typical objectives of unit testing are to:

• Enable the identification of unit-level defects by causing corresponding failures to occur.
• Identify defects that are not easily identified during other kinds of testing.
• Enable the developer to confidently iterate and refactor the software unit, knowing that any defect will be quickly discovered during regression testing.
• Enable the developer to determine if the unit:
  • Is complete.
  • Fulfills its responsibilities and does not violate its assertions (the oracles for unit testing).
  • Works as designed.
  • Is ready to be submitted for integration as part of a code build.

Preconditions

Unit testing can typically begin when the following preconditions hold:

• The unit responsibilities have been determined.
• The unit's assertions (i.e., class invariant, method preconditions and postconditions) have been specified.
• The unit interface has been designed.
• The development environments are ready.

Completion Criteria

Unit testing is typically complete when the following postconditions hold:

• A complete test suite of test cases exists for every:
  • Public software unit.
  • Domain class and interface.
• Object-oriented classes. Unit testing is complete if a minimal test suite of test cases that meet the following coverage criteria has been successfully ran for each class:
  1. Each test should evaluate the actual versus expected:
     • Value (if any) returned by the object under test (OUT).
     • Outbound exceptions (if any) that can be raised by the OUT.
     • State of the OUT.
     • Messages (if any) to be sent by the OUT.
     • Inbound exceptions (if any) handled by the OUT.
  2. Blackbox test coverage criteria:
     • Every responsibility has at least one test case.
     • Every individual assertion (i.e., precondition, postcondition, and invariant) has at least two test cases (one for true and one for false).
     • Every public operation (method) of the class under test that is not a pure getter or setter has at least one test case. This includes tests for:
       • Newly defined operations are developed.
       • Unmodified inherited operations are rerun (regression testing).
       • Overridden inherited operations are iterated or developed.
     • Every constructor has at least one test case.
     • State based testing. At least one test case for every public operation in every state (i.e., state of the object under test, state of every message parameter, state of every collaborator, and state of every exception handled) ensures coverage of every transition from every state.
  3. Whitebox test coverage criteria:
     • Statement coverage (absolute minimum used for simple classes only).
     • Branch coverage (default for average classes).
     • Condition/branch coverage (for complex, defect-prone classes).
     • Every define/use path for every variable attribute.
• Procedural functions. Unit testing is complete if:
  • A test suite containing test cases exists for:
    • Every state of every parameter.
    • Every basis path through the function has been tested.
• HTML webpages. Unit testing is complete if:
  • Tag testing (e.g., using WebLint) is performed to determine if the HTML is syntactically correct.
  • A test suite containing test cases exists for:
  • Every link (to determine if the link is broken).
  • Every parameter state of every input parameter.
  • Every client side code fragment (e.g., buttons that only affect the client and do not communicate with the server).
• These test suites execute properly.
• No failures are reported.

Tasks with Techniques

Unit testing typically involves performing the following testing tasks using the following techniques:

• Test Planning
• Test Reuse
• Test Design:
  • Assertions and exceptions
  • Boundary-value testing
  • State based testing (e.g., transition tree)
  • Structured (basis path) testing using McCabe's cyclometric complexity measure
  • Error guessing
  • Decision tables
  • Abstract class via concrete test subclass
  • Interface via concrete class
  • Dependency-based testing (i.e., test collaborators and superclasses before testing class that depends on them)
  • Parallel test driver hierarchy
  • Embedded test suite
  • Test subclasses to test message
  • Incremental testing (design test cases incrementally as classes are incrementally developed, a method at a time)
  • Test first (i.e., design test cases before designing method implementations)
  • Peer testing (pair programmers codevelop tests or one develops tests while other develops class under test).
• Test Implementation:
  • Test first
  • Peer testing
• Test Execution:
  • Automated testing
  • Regression testing
  • Peer testing
• Test Reporting:
  • Automatic report generation (by tool)

Environments

Unit testing is typically performed on the following environments using the following tools:

• Engineering Environment (initial and regression testing):
  • Test harness (driver) tools such as Junit
  • Static analysis tools such as WebLint
  • Whitebox test coverage tools
• Integration Environment (regression testing):
  • Test harness (driver) tools such as Junit
  • Static analysis tools such as WebLint
  • Whitebox test coverage tools

Phases

Unit testing typically consists of the following tasks being performed during the following phases:

Guidelines

• Develop one or more test models (e.g., state based) that partition all possible combinations of preconditions and test stimuli into equivalence sets (i.e., into disjoint sets, each element of which should produce the equivalent test results).
• Use blackbox testing as the primary testing approach.
• Use whitebox testing as a secondary testing approach to achieve coverage criteria that was not totally achieved by blackbox testing.
• Use a tool to determine the level of whitebox coverage achieved by the test suite.
• Because a single test case can help achieve multiple test criteria, the test suite for a class should avoid having redundant test cases.