Developmental/Acceptance Validation

For systems that have not yet been developed, the steps required for a developmental and acceptance validation should be included in the work plan for the system devel­opment project. The critical steps that should be included are to 1) develop and maintain a comprehensive test plan throughout the development project; 2) develop test proto­cols; test scripts, and test cases; 3) conduct formal design and programming verification, integrated system testing, and acceptance testing; 4) document and archive test results, and 5) develop and maintain SOPs.

Most importantly, at the end of the project, formal valida­tion documentation should be assembled, which includes or references all controlled system documentation, SOPs, test plans, and test results. All validation documentation
should be put under formal document control, with an SOP for keeping it current with future changes to the system. It is the existence and ongoing maintenance of the validation documentation that provides evidence of system quality and reliability.

Retrospective Validation

For systems that are already in live production, there may be no choice but to conduct a retrospective validation. The key to keeping the cost of retrospective validation within reason is to select the right level of scrutiny needed for each existing system This is necessary so that the validation is limited to only those systems or functions that are critical from a regulatory perspective. The steps are as follows:

1. Take inventory of existing software. Identify all pack­aged application software, custom systems, utility software, and operating systems.

2. Rank software by priority for validation. This can be done by performing a risk analysis, which analyzes the worst case result if the software fails. Systems in which software errors could potentially result in hu­man death or severe injury obviously would have the highest priority for validation (e.g., software driving an X-ray system). Systems that have no such effect would not need to be validated (e.g., E-mail software). Most systems would fall somewhere between these two extremes.

3. Establish validation teams. Include users of the sys­tem, developers of the system, and independent re­viewers.

4. Perform the validation by reviewing existing SOPs and any existing system documentation, reviewing source code, and interviewing knowledgeable users of the system. From these, a complete description of the system and set of SOPs should be written or as­sembled. Continue by writing and executing a test plan with comprehensive test protocols, test scripts, and test cases, as described earlier. Make any neces­sary corrections and perform additional testing as needed.

5. Assemble the validation documentation as described earlier.

6. Obtain third party audit and certification as required.

Cooperation Needed from Software Vendors

As discussed earlier, validation is the responsibility of management and users, not the software vendor. Never­theless, the software vendor plays an important role in validation. First, the vendor can subject his own software development operations to a third party audit. This formal certification by a third party can then be used as part of the validation documentation for a company's developmental, acceptance, or retrospective validation. The vendor's vali­dation cannot substitute for a company's own validation, but it can supplement it as an assurance that the package was developed in a reliable fashion.

Second, the vendor can supply you with the source code for the package. Although this sometimes is a problem from a contractual or trade secret standpoint, source code avail­ability is critical for two reasons. It.is needed to do "white box" testing, as described earlier, and it is needed for long term support of a package in the event that the software vendor ever goes out of business. For these reasons, companies should take measures during contract negotia­tions to ensure access to source code.

Caveat

This paper is meant to serve only as a brief introduction to the subject of software validation. It is not meant to, and cannot, serve as a complete guide for satisfying regulatory requirements. Regulated companies should refer to rel­evant governmental agency publications for such guidance.