Operations Manual Testing

Definition

Operations manual testing is the system testing of an integrated blackbox application against its operational requirements.

The typical goals of functional testing are to:

Cause the application to fail when ported to its various environments so that the underlying defects can be identified, analyzed, fixed, and prevented in the future.

The typical objectives of operations manual testing are to:

Cause failures concerning the operational requirements that help identify defects that are not efficiently found during other types of testing.
Report these failures to the development teams so that the associated defects can be fixed.
Help determine the extent to which the system is ready for launch testing.
Help provide project status metrics (e.g., percentage of operational requirements successfully tested).
Provide input to the defect trend analysis effort.

Typical examples include operations manual testing of an application that is to be deployed to a different environment including:

Hardware platform (including clients, servers, network connectivity devices, input devices, and output devices).
Operating system (including versions and service packs).
Browser (including both types and versions).

Operational test execution can typically begin when the following preconditions hold:

The operational requirements to be tested have been specified and fully implemented.
The relevant software components have passed unit testing.
Software integration has occurred (i.e., some operations manual testing can begin prior to the distribution of the software components onto the hardware components).
The relevant system components have passed independent integration testing.
The independent test team is adequately staffed and trained in operations manual testing.
Each required production environment is ready.

Operations manual testing is typically complete when the following postconditions hold:

The application has been successfully run on each required production environment.
The operational test suites execute successfully in each required production environment.

Operations manual testing typically involves the independent test team performing the following testing tasks:

Operations manual testing is typically performed on the following environments using the following tools:

Operations manual testing typically results in the production of all or part of the following work products from the test work product set:

Documents:
- System Requirements Specification (operational requirements)
- Project Test Plan
- Master Test List
- Test Procedures
- Test Report
- Test Summary Report
Software and Data:

Operations manual testing typically consists of the following tasks being performed during the following phases:

PHASE → TASK ↓	Business Strategy	Business Optimization	Initiation	Construction	Delivery	Usage	Retirement
Test Planning	Not Applicable	Not Applicable	Completed	Optional Regression	Not Applicable	Not Applicable	Not Applicable
Test Reuse	Not Applicable	Not Applicable	Optionally Started (*)	Completed	Not Applicable	Not Applicable	Not Applicable
Test Design	Not Applicable	Not Applicable	Optionally Started (*)	Completed	Not Applicable	Optional Regression	Not Applicable
Test Implementation	Not Applicable	Not Applicable	Optionally Started (*)	Completed	Not Applicable	Optional Regression	Not Applicable
Test Execution	Not Applicable	Not Applicable	Optionally Started (*)	Completed	Not Applicable	Optional Regression	Not Applicable
Test Reporting	Not Applicable	Not Applicable	Not Applicable	Completed	Not Applicable	Optional Regression	Not Applicable

(*) Optional operations manual testing of the executable architecture as well as the COTS components during the vendor and tool evaluation and vendor and tool selection tasks.

The iterative and incremental development cycle implies that operational testing is regularly performed in an iterative and incremental manner.
Operations manual testing must be automated if adequate regression testing is to occur.