Test Report

Definition

A test report is a testing work product that formally reports the results of the execution of a test suite of test cases.

Objectives

The typical objectives of a test report are to:

• Summarize the test that was performed.
• Report any failures that were observed.
• Report any test suites or test cases that could not be performed.

Benefits

The typical benefits of a test report includes:

• The reported failures can be used to identify and repair the associated defects.
• The system and launch test results can be used to perform trend analysis in order to determine when the application should be placed into production.
• The reported failures can be used to identify potential problems with:
  • The development process so that it can be tailored and the OPF can be improved.
  • Staff training so that additional training and mentoring can be provided.

Contents

The typical contents of a test report include:

• Test Summary (type of test, requirements tested, test suites executed)
• Failures Observed (test suite, test case, expected results, and actual results)
• Test Failures Observed (test suites and test cases that could not be performed and the reason why)
• Appendices:
  • Major Issues
  • TBDs
  • Assumptions

Stakeholders

The typical stakeholders of a test report include:

• Producers:
  • The Software Development Team for unit tests and low-level software integration tests.
  • The Integration Team for high-level software integration tests and system integration tests.
  • The Independent Test Team for most system and launch tests.
  • The Security Team for security test.
  • The User Experience Team for usability tests:
• Evaluators:
  • The Test Inspection Team on a spot-check basis.
• Approvers for system and launch test procedures:
  • Technical Leader
  • Project Manager
  • Customer Representative
• Maintainers:
  • The Integration Team for high-level software integration tests and system integration tests.
  • The Independent Test Team for most system and launch tests.
  • The Security Team for security test.
  • The User Experience Team for usability tests:
• Users:
  • The Project Managment Team uses the system and launch test reports to perform trend analysis in order to:
    • Estimate when the application will be ready for production.
    • If the application requires minor or major defect removal prior to being placed into production.
    • If the application is ready to be placed into production (including defect related risks).
  • The Project Team uses the failures identified in the test reports to identify and repair the associated defects.
  • The Process Team uses the test reports to identify potential problems with:
    • The development process so that it can be tailored and the OPF can be improved.
    • Staff training so that additional training and mentoring can be provided.
  • The User Experience Team uses the test reports for usability tests to identify and repair the usability defects.

Phases

• Business Strategy: Not Applicable
• Business Optimization: Not Applicable
• Initiation: Started
• Construction: Completed
• Delivery: Maintained
• Usage: Maintained
• Retirement: Archived

Preconditions

A test report typically can be started if the following preconditions hold:

• The Project Test Plan has been completed.
• A associated test has been performed.

Inputs

The typical inputs to a test report include:

• Work Products:
  • Project Test Plan
• Stakeholders:
  • None

Guidelines

• A test report is mandatory for each system test, which may include multiple test suites of test cases.
• Different kinds of system testing typically occur during different system tests, each of which results in its own test report.
• The Project Test Plan strongly influences the number of test reports that will be produced.
• The format of the test report is flexible. If a test tool is used to perform testing and the tool automatically produces reports, then these reports can be used as is and need not be transcribed into the format of the Test Report content and format standard and template.
• It is not the responsibility of the test team to identify the defect that causes the failure or to recommend how the defect should be repaired. The test team need only document the failure adequately so that the team responsible for repairing the defect can repeat the failure and localize and identify the defect.
• Test reports are not updated unless a defect is found in the report; repeated tests (regression testing) result in new test reports.
• Electronic copies of system test reports should be maintained at least until delivery (and possibly for a period afterwards in order to prove that the system was adequately tested.

Conventions

A test report is typically constrained by the following conventions:

• Content and Format Standard
• Template
• Inspection Checklist

Examples

• Example Test Report