Validation Is Not Testing: Why the Distinction Matters in Regulated Software
A passing test suite is not evidence of fitness for use
A software build can pass unit, integration, security and end-to-end tests, yet still be unsuitable for use in a regulated operation. This is not a criticism of testing. It reflects the different question that validation must answer.
Testing asks whether specified functions behave as expected under defined conditions. Validation asks whether the complete system consistently fulfils its intended use within the environment in which it will operate. The latter includes people, procedures, equipment, interfaces, data, infrastructure, security controls and operational constraints. It also includes the consequences of failure.
For a calibration management platform, for example, a test may establish that an overdue-status rule correctly identifies an asset after its due date. Validation must establish more: that dates are sourced correctly; time zones and clock behaviour are understood; authorised users can review and act on the status; notifications are delivered or their failure is detected; records are retained; access is controlled; and the process remains effective during foreseeable degraded conditions. The feature may be technically correct while the operating process is not.
Verification and validation address different evidence
The commonly used distinction is straightforward. Verification provides evidence that a system has been built according to specified requirements. Validation provides evidence that the system, as deployed and operated, is suitable for its intended use.
In practice, the boundary is not always clean. A well-designed integration test may support both verification and validation evidence. However, treating test execution as the entire validation programme creates predictable gaps.
Automated tests are particularly valuable for repeatability. They can exercise normal flows, boundary conditions, error handling and regressions at a frequency that manual execution cannot match. They should be part of a controlled engineering lifecycle, with reviewed requirements, version-controlled code, reproducible builds and traceable test results. Their limitations are equally important. A test only demonstrates what its design, input data, assertions and environment make observable.
It cannot establish that a requirement was complete, that the configured production environment matches the tested one, or that users can perform a controlled process without introducing avoidable error. Nor can it establish that a third-party service will meet its assumed availability, retention or identity-management behaviour over time.
Intended use defines the validation problem
Validation begins before test cases are written. The intended use must be sufficiently specific to identify what the system is expected to do, who will use it, which decisions or records depend upon it, and what constitutes an unacceptable failure.
A system used to assist internal planning has a different validation burden from one that creates quality records, supports release decisions, maintains calibration status or supplies data to a regulated submission. The technology may be similar. The required evidence is not.
This is why generic statements such as “the application manages records securely” are inadequate as validation requirements. They need to be decomposed into observable, testable and reviewable behaviours. For example: which roles may create, amend, approve or retire a record; how identity is established; which changes require review; what data must be retained; how corrections are represented; and whether the system preserves the ability to reconstruct the history of a record.
Requirements should also state relevant non-functional properties. Performance limits, recovery objectives, auditability, access control, backup restoration, interface failure handling and data migration controls may all affect fitness for use. A system that produces correct records but cannot recover them within an operationally acceptable period is not necessarily fit for its purpose.
The production environment is part of the system
In regulated use, the system boundary must extend beyond application code. It commonly includes cloud services, operating systems, databases, identity providers, network controls, configured workflows, reporting tools and connected instruments or data sources.
Configuration deserves particular attention. A validated application can be rendered unsuitable by an uncontrolled role assignment, altered workflow, changed retention setting or modified interface mapping. Controlled change is therefore not an administrative addition made after delivery. It is part of maintaining the validated state.
The same applies to data. Test data may prove that a calculation executes correctly, but validation must consider the origin, integrity and interpretation of operational data. Where measurements are involved, raw values alone may not be defensible. Calibration status, instrument identification, units, timestamps, uncertainty, transformation rules and source provenance can materially affect whether a result can support a decision.
Production-like qualification environments, controlled configuration baselines and documented release procedures reduce the gap between demonstrated behaviour and deployed reality. They do not remove the need for judgement, but they make that judgement supportable.
Evidence must support reconstruction and change
A credible validation record links intended use, risk assessment, requirements, design decisions, test evidence, deviations, approvals and released configuration. Traceability is not valuable because a matrix exists. It is valuable because it allows an organisation to answer practical questions after an event: what was required, what was implemented, what was tested, what changed, and whether the change affected the basis for use.
Deviations require the same discipline. A failed test that is corrected and rerun is not automatically closed by the later pass result. The organisation needs to understand the cause, assess the impact on other functions and records, and determine whether requirements, risk controls or additional testing need revision.
Risk-based validation does not mean testing only the functions considered convenient or likely to fail. It means applying effort in proportion to patient, product, data-integrity, operational and compliance consequences, while retaining a defensible rationale for scope and depth. The applicable regulatory framework, organisational procedures and intended use determine the precise expectations. Authoritative requirements should be confirmed against the relevant regulator, standard and contractual obligations rather than inferred from a generic checklist.
Software testing establishes that important behaviours have been exercised. Validation establishes the justified confidence required to operate the whole system for its stated purpose. In regulated environments, that distinction is where technically functional software becomes evidence-supported, controlled software suitable for real decisions.