Security Testing

Definition

Security Testing
(a.k.a., Penetration Testing)

the testing of a system, application, or component against its security requirements and the implementation of its security mechanisms

Goals

The typical goals of security testing are to:

• Cause the system, application, or component to fail:
  • Failure to meet security-related requirements (blackbox testing)
  • Failure to properly implement security mechanisms (whitebox/graybox testing)
• Thereby enable the underlying defects can be identified, analyzed, fixed, and prevented in the future.

Objectives

The typical objectives of security testing are to:

• Requirements.
  Partially verify the application (i.e., to determine if it fulfills its security requirements):
  • Identification
  • Authentication
  • Authorization
  • Content Protection
  • Integrity
  • Intrusion Detection
  • Nonrepudiation
  • Privacy
  • System Maintenance
• Mechanisms.
  Determine if the system causes any failures concerning its implementation of its security mechanisms:
  • Encryption and Decryption
  • Firewalls
  • Personnel Security:
    • Passwords
    • Digital Signatures
    • Personal Background Checks
  • Physical Security:
    • Locked doors for identification, authenication, and authorization
    • Badges for identification, authenication, and authorization
    • Cameras for identification, authenication, and authorization
• Cause Failures.
  Cause failures concerning the security requirements that help identify defects that are not efficiently found during other types of testing:
  • The application fails to identify and authenticate a user.
  • The application allows a user to perform an unauthorized function.
  • The application fails to protect its content against unauthorized usage.
  • The application allows the integrity of data or messages to be violated.
  • The application allows undetected intrusion.
  • The application fails to ensure privacy by using an inadequate encryption technique.
• Report Failures.
  Report these failures to the development teams so that the associated defects can be fixed.
• Determine Launch Readiness
  Help determine the extent to which the system is ready for launch.
• Project Metrics.
  Help provide project status metrics.
• Trend Analysis.
  Provide input to the defect trend analysis effort.

Examples

Typical examples include:

• Penetration Testing

Preconditions

Security test execution can typically begin when the following preconditions hold:

• The security requirements to be tested have been specified and fully implemented.
• The corresponding security mechanisms have been implemented.
• The relevant software components have passed unit testing.
• The relevant system components have passed system integration testing.
• The security team is adequately staffed and trained in security testing.
• The test environment is ready software security testing.
• The production environment are ready for regression, hardware, and physical security testing.

Completion Criteria

Security testing is typically complete when the following postconditions hold:

• A test suite of test cases exists for each security requirement.
• The test suites for every scheduled security requirement execute successfully.
• Specifically:
  • Physical security is determined to be effective.
  • Major software components and their code have been reviewed for security vulnerabilities, and fixes have been made and verified.
  • The hardware architecture has been reviewed for security vulnerabilities, and fixes have been made and verified.
  • Ports have been scanned, fixes have been made, and a final scan reveals no security vulnerabilities.
  • Network traffic has been analyzed, fixes have been made, and a final analysis reveals no security vulnerabilities.

Tasks

Security testing involves the security team performing the following testing tasks:

• Test Planning
• Test Reuse
• Test Design:
  • Performs security review of major software components and their code.
  • Performs security review of hardware architecture (production environment) for hardware placement, network addressing and segment, and application distribution.
• Test Implementation
• Test Execution:
  • Tests physical security.
  • Scans ports to identify host-level and network-level vulnerabilities.
  • Analyzes traffic to identify IP information about potential attackers.
  • Regression Testing
• Test Reporting

Environments

Security testing is performed on the following environments using the following tools:

• Test Environment:
  • Port scanners (e.g., ISS Internet Security Scanner and ISS DB Scanner), which are used to identify open (listening) ports to identify vulnerabilities. Port scanners are either host-level (OS-specific) or network-level (routers, firewalls, and service/shares on networked machines).
  • Traffic analyzers (e.g., Etherpeek) are used to capture packets moving in and out of specific network cards to identify IP information about potential attackers.
• Production Environments:
  • Port scanners (e.g., ISS Internet Security Scanner and ISS DB Scanner), which are used to identify open (listening) ports to identify vulnerabilities. Port scanners are either host-level (OS-specific) or network-level (routers, firewalls, and service/shares on networked machines).
  • Traffic analyzers (e.g., Etherpeek) are used to capture packets moving in and out of specific network cards to identify IP information about potential attackers.

Work Products

Security testing typically results in the production of all or part of the following work products from the test work product set:

• Documents:
  • Project Test Plan
  • Master Test List
  • Test Procedures
  • Test Report
  • Test Summary Report
• Software and Data:
  • Test Harness
  • Test Scripts
  • Test Suites
  • Test Cases
  • Test Data

Phases

Security testing typically consists of the following tasks being performed during the following phases:

(1) Optional security testing of COTS software components during the technology analysis and technology vendor selection tasks.

(2) The first round of security testing occurs after the hardware components have been networked, and the hardware architecture has been reviewed for security considerations.

(3) The second round of security testing occurs after both the software components and the content has been distributed on the hardware components.

(4) The third round of security testing is part of the acceptance testing prior to launch.

(5) Security testing should be performed on a periodic basis thereafter.

(6) Optional security testing of the executable architecture as well as the COTS components during the vendor and tool evaluation and vendor and tool selection tasks.

Guidelines

• Security testing typically tests systems, applications, and components (e.g., hardware, software, data, personnel, and procedure). It can also be used to test centers and facilities.
• Security testing may be either blackbox testing (against requirements) or whitebox/graybox testing of security mechanisms.
• Security testing requires the tester to take on the mindset of the attacker and to use the tools and techniques of the attacker. Security testing therefore requires that the testers are authorized to attack for testing purposes.
• Security tests may be either automated using a security tool or test harness or else performed manually (e.g., tests of physical security).
• The scope of security testing includes testing the fulfillment of:
  • Security Requirements
    (Because security requirements are a kind of quality requirements, testing of these requirements is a kind of non-functional testing.)
  • Security-Significant Requirements
    (Because these are non-security requirements with security ramifications such as certain functional, data, and interface requirements, testing of these requirements typically involves some functional testing.)
  • Requirements for Security Subsystems
    (For example, the testing of access control and encryption/decryption subsystem includes all types of testing including functional, performance, reliability, etc.)
  • Security Constraints
    (This typically includes multiple types of whitebox and graybox testing.)
• Security testing should detect the following security failures:
  • The system fails to identify and authenticate a user.
  • The system allows a user to perform an unauthorized function.
  • The system fails to protect its content against unauthorized usage.
  • The system allows the integrity of data or messages to be violated.
  • The system allows undetected intrusion.
  • The system fails to ensure privacy by using an inadequate encryption technique.
• In a sense, security testing is never complete because security testing should be repeated on a regular basis.
• Batch scripts can be applied to perform registry fixes.