Test Evaluation

Definition

Test evaluation is the quality control task during which the test work products are evaluated.

Objectives

The typical objectives of the test evaluation task are to:

• Determine if the deliverable testing work products are:
  • Correct.
  • Complete.
  • Consistent (internally, externally with other work products, and externally with related conventions).
  • Understandable.
• Identify defects in the deliverable testing work products so that:
  • The defects can be fixed.
  • Defect trend analysis can be performed to improve the process and staff training.
• Determine if the testing tasks are:
  • Completed.
  • Effective.
  • Efficient.

Preconditions

The test evaluation task can typically begin when the following preconditions hold:

• The construction phase has started.
• The test inspection team is adequately:
  • Staffed.
  • Trained in the test evaluation task.
  • Trained in its associated techniques (e.g., inspection).
• At least one testing work product is ready for evaluation.

Completion Criteria

The test evaluation task is typically complete when the following postconditions hold:

• The following testing work products have passed their evaluations (e.g., inspection):
  • Project Test Plan
  • Master Test List
  • Test Procedure
  • Test Report
  • Test Summary Report
  • Test Harness
  • Test Script
  • Test Suite
  • Test Case
  • Test Data

Steps

The test evaluation task typically involves the following teams performing the following steps in an incremental, iterative, parallel manner:

• The various testing teams informally evaluate their work products in accordance with the associated technique used.
• The test inspection team formally evaluates the work products in accordance with the associated technique(s) used.

Techniques

The test evaluation task typically can be performed using the following techniques:

• Quality Evaluation Techniques:
  • Desk Checking
  • Inspecting
  • Quality Metrics
  • Static Analysis
  • Walkthroughs

Work Products

The test evaluation task typically results in the production of the following work products:

• Quality Set:
  • Inspection Report
  • Inspection Summary Report

Guidelines

• Inspecting tends to be the most effective technique for identifying defects.
• Evaluations can be incremental and are typically iterative.
• Because defects can occur in the test work products, it is important to ensure that tests do not produce any false positive or false negative results.
• The iterative and incremental nature of most development cycles means that regression testing happens more often and thus becomes more important. Test evaluation helps ensure that the repeated tests do not contain defects that will continually cause test failures.