Verification vs Validation: Assess Software Evidence

Verification and validation (V&V) are two distinct processes for assessing software evidence: verification answers “Did we build the product right?” by checking that artifacts meet their specified requirements, while validation answers “Did we build the right product?” by evaluating whether the software fulfills its intended use in the real world. To assess software evidence effectively, you must collect objective proof from both processes—such as test results, inspection records, and user feedback—and judge that proof against the relevant requirements and operational context. This article explains the difference, the types of evidence you need, and how to evaluate that evidence systematically.

What Are Verification and Validation?

Verification and validation are two complementary quality assurance activities defined in ISO/IEC/IEEE 12207 and other software engineering standards. Verification focuses on consistency: it checks that each artifact (requirements, design, code, test cases) conforms to the artifact that preceded it. For example, verifying that the code implements the design, or that the design traces back to the requirements. Validation focuses on value: it checks that the final product meets the stakeholder’s actual needs and works in the intended environment. A classic way to remember the difference is: verification is “doing the thing right,” validation is “doing the right thing.”

Both processes generate evidence—records, logs, reports, and artifacts that demonstrate the software’s quality. Assessing that evidence is critical for regulatory compliance (e.g., FDA medical device software, DO-178C avionics), for safety-critical systems, and for any project where defects have high cost. Without a systematic approach, teams often mistake activity for evidence: running many tests does not prove quality if the tests are weak or the results are not reviewed.

Types of Software Evidence

Software evidence can be grouped into four broad categories:

• Process evidence: records that the development process was followed—e.g., code review logs, sprint burndown charts, tool configuration files.
• Product evidence: results from testing, static analysis, and inspections—e.g., unit test reports, code coverage percentages, defect tracking data.
• Operational evidence: data from the running system—e.g., performance benchmarks, error logs, uptime monitoring, user analytics.
• Stakeholder evidence: feedback from users, acceptance test sign-offs, and usability study results.

Each type answers a different question. Process evidence shows that you did the work; product evidence shows that the work was correct; operational evidence shows that the software behaves in production; stakeholder evidence shows that it meets real needs. For high-assurance systems, you typically need all four.

How to Assess Verification Evidence

Verification evidence is assessed by checking traceability and coverage. Start by mapping each requirement to the design elements, code modules, and test cases that implement and test it. Then evaluate whether the evidence is sufficient:

1. Check traceability: Every requirement should trace forward to code and tests, and every test should trace back to a requirement. Use a traceability matrix or a tool like Jira or Polarion.
2. Evaluate test coverage: Look at code coverage (statement, branch, condition) and requirement coverage. High code coverage (e.g., 80%+ branch) is good, but it does not guarantee that all behaviors are tested.
3. Inspect static analysis results: Static analyzers (e.g., SonarQube, Coverity) find defects that tests might miss. Review the severity and whether the team fixed or justified each finding.
4. Review inspection records: Code reviews and design reviews produce evidence of human scrutiny. Check that reviews were performed by qualified peers and that all comments were resolved.

A common mistake is to rely solely on automated test results. Automated tests are valuable, but they only prove that the software behaves as the test cases expect—not that the test cases themselves are correct. Pair automated results with manual inspections and requirement-based test design.

How to Assess Validation Evidence

Validation evidence is harder to assess because it involves subjective and contextual factors. The goal is to determine whether the software will work for the intended users in the real environment. Key evidence sources include:

• User acceptance testing (UAT): Users execute real workflows and confirm the software meets their needs. Look for signed-off UAT scripts and defect reports from UAT.
• Beta testing and pilot runs: Data from a limited deployment can reveal issues that laboratory testing misses—e.g., integration problems, performance under real load, usability friction.
• Field monitoring: After release, track error rates, crash reports, and user feedback. A low crash rate and positive feedback are validation evidence.
• Usability studies: Observations of users performing tasks can uncover design flaws that do not appear in functional tests.

To assess validation evidence, ask: Does this evidence come from a representative environment? Are the users representative? Were the tasks realistic? A beta test with five internal employees is weaker evidence than a pilot with fifty external customers. Also consider the duration: a one-hour usability test is less convincing than a two-week field trial.

Common Problems and Misconceptions

One widespread misconception is that verification and validation are interchangeable, or that “testing” covers both. In reality, testing is just one technique for gathering evidence; verification also includes inspections and static analysis, while validation includes user feedback and operational monitoring. Another misconception is that passing all tests means the software is validated—but tests are only as good as their assumptions about the environment. A system might pass every test yet fail in production because the test environment did not match the real network latency or data volume.

Other common problems include:

• Incomplete traceability: Teams often lose traceability when requirements change. Without a current traceability matrix, you cannot prove that all requirements are verified.
• Coverage theater: Chasing 100% code coverage can lead to trivial tests that add no value. Focus on meaningful coverage of critical logic.
• Ignoring negative evidence: Teams sometimes hide or downplay defects. A transparent defect log with severity and resolution status is essential evidence.
• Confusing activity with evidence: Holding many meetings or writing long documents does not prove quality. Evidence must be objective and verifiable.

To avoid these, establish a clear evidence management plan at project start: define what evidence will be collected, who is responsible, and how it will be reviewed.

How to Choose the Right Assessment Approach

The depth of your V&V assessment should match the risk and criticality of the software. For a small internal tool, a simple checklist and a few smoke tests may be enough. For a medical device or an aircraft control system, you need rigorous, documented evidence that follows industry standards. Consider these factors:

• Regulatory requirements: If your software is regulated (FDA, FAA, EU MDR), you must follow specific standards and produce evidence that auditors will accept.
• Cost of failure: If a defect could cause injury, financial loss, or data breach, invest more in verification and validation.
• Complexity: Highly complex systems with many interactions require more thorough testing and analysis.
• Stakeholder expectations: If users are paying for a critical service, they expect evidence that it works.

A practical approach is to use a risk-based testing strategy: allocate more effort to the features that carry the most risk. For each risk, define the evidence that would mitigate it, then execute the activities that produce that evidence.

Frequently Asked Questions

What is the difference between verification and validation in simple terms?

Verification checks that the software meets its written specifications (e.g., “does the login page accept a 10-character password?”). Validation checks that the software meets the user’s actual needs (e.g., “does the login page allow users to reset their password easily?”). Verification is about conformance to documents; validation is about fitness for use.

Can a software pass verification but fail validation?

Yes, absolutely. A system can be built exactly to its requirements (verification passes) but still not solve the user’s problem because the requirements were wrong or incomplete. For example, a banking app might correctly implement a transfer feature, but if users cannot find the button, it fails validation. This is why both processes are necessary.

What is the best evidence for validation?

The strongest validation evidence comes from observing real users in a real environment performing real tasks. This includes user acceptance test sign-offs, beta test results, production monitoring data (e.g., low error rates, high user retention), and direct user feedback. The more representative and longer the observation, the stronger the evidence.

How much test coverage is enough?

There is no universal number. For safety-critical systems, standards like DO-178C require specific coverage levels (e.g., MC/DC for the highest level). For general software, a pragmatic target is 70-80% branch coverage, but the real goal is to cover all critical behaviors and edge cases. Coverage is a means, not an end—always review which tests are actually meaningful.

What is a traceability matrix and why is it important?

A traceability matrix is a table that links requirements to design, code, and tests. It is important because it provides a clear, auditable map showing that every requirement is addressed and every test has a purpose. Without it, you cannot prove that your verification is complete, and you risk missing gaps that could lead to defects.

How do you handle evidence when requirements change?

When requirements change, you must update the traceability matrix and re-run the affected tests. For verification, you need to show that the new requirement is implemented and tested. For validation, you may need to re-confirm with users that the change still meets their needs. Keep a version history of all evidence to show the evolution.

Related

• iPhone Battery Settings: 12 Tweaks That Extend Life
• How to Record with a Webcam on Windows and Mac
• How to Stop Alexa from Recording Your Conversations