Black Box and White
testing, two categories of verification can be used. The first is a
black box test, or functional test. The tester designs test cases
based on functional specifications. The tester is not concerned with
how the program accomplishes the specification. He treats the
program like a black box. The second method is a white box test, or
structural test. The tester designs test cases based on detailed
examination of internal program code and system structure. The test
method is to exercise as many paths with the program and system as
possible, with the goal of causing an error. White box testing is
more extensive than black box testing, but it requires that the
person preparing the test protocol have knowledge of the program's
internal structure. Research has shown that the most successful
software testing is a combination of black box and white box testing
Complete Range of
test protocols, it is important that as many conditions as possible
be tested. This includes normal cases (data that is normally
expected), invalid cases (data that must be rejected), and boundary
cases (data that is marginally valid or invalid). Test cases should
also test for exceptional data (e.g., special characters) and cases
where no data is entered (empty files, restart conditions, etc.).
Test cases should also include missing paths (i.e., the user omits a
required step), wrong paths (i.e., the user executes steps in the
wrong sequence), and wrong actions (e.g., the user attempts
"unreasonable" actions or actually tries to "break the system").
System and Acceptance Testing
In addition to the
program verification methods described above, a number of additional
tests should be performed during the Integrated System Test or the
Acceptance Test phases. These include, but are not limited to, the
1. A. volume test
exercises the system with real-world or a maximum amount of data.
This is necessary because previous testing may have only involved
small amounts of test data. The load / stress test evaluates the
system's ability to handle overload conditions.
2. Human factor
testing is designed to ensure that the system/user interface leads
the operator to correct understanding and use. This would include
testing of the SOPs and all system documentation, perhaps with users
that were not previously involved in the system development.
3. A. performance
test measures the ability of a system to perform its function in an
acceptable manner (e.g. acceptable response time and throughput).
4. Security testing determines to what extent the system can detect
and prevent unauthorized access or corruption of data. This
includes the ability of the system to detect or prevent the
introduction of computer viruses.
audit is the process of verifying that all hardware and software
components have been installed at the correct version, that the
technical documentation is complete and accurate, and that all
change requests have been resolved.
6. Recovery testing
involves testing of the disaster recovery plan to ensure that the
system can recover from data loss, data damage, or system failure,
within an acceptable time frame.
7. A reliability
assessment is the process of determining the achieved level of
reliability for a system, measured as mean-time-between-failure (MTBF),
number of failures or errors per period, maintenance man-hours per
period, or some other measure of reliability. The reliability
assessment can begin during acceptance testing and should be
continued throughout the Operation and Maintenance phase.
8. A robustness
assessment is an evaluation of software to determine how well it can
continue to operate despite introduction of invalid inputs, power
failures, or other exceptional conditions.
Organizing the Validation Effort
There are three
types of system validation. Developmental validation is done for a
new system during its development. This is the easiest form of
validation because it can be done as part of normal system
development. Developmental validation is consistent with the total
quality concept of "designed in" quality. Acceptance validation is
performed for a new or modified system as part of the implementation
or "go live" process. Retrospective validation is performed when an
unvalidated system is already in live use. This is the process of
going back and doing what should have been done when the system was
first developed. Retrospective validation is an after-the-fact
attempt to "inspect in" quality.
To Be Continued