Assurance vs Validation

A healthcare technology company onboarded a clinical data management vendor whose SOC 2 Type II report covered access control, availability, and confidentiality criteria. The assurance from the SOC 2 was cited in the TPRM assessment as evidence that the vendor's access controls operated effectively. What the TPRM assessment did not verify was the specific question the healthcare company's CISO had asked: can a user in the vendor's system access any patient's record, or are access controls scoped to the specific patients each user is authorised to treat? The SOC 2 confirmed that the vendor's access control programme operated , users were authenticated, access was logged, and access reviews were conducted quarterly. It did not test whether the authorisation model prevented cross-patient record access by an authenticated user. That was not a criterion in the SOC 2 framework. During a penetration test the healthcare company commissioned for the vendor (exercising their right-to-audit), the tester authenticated as a standard clinical user and successfully accessed records for patients outside their authorised caseload through an API parameter manipulation that the vendor's access control programme had not addressed. The SOC 2 assurance was valid. The specific authorisation control the healthcare company needed validated had not been tested by the SOC 2.

What is the Assurance vs Validation Problem, Really?

Assurance is professional confidence, expressed by an independent third party, that a system of controls is operating in accordance with defined criteria based on evidence gathered through structured assessment methods. SOC 2 assurance, ISO 27001 certification, and PCI-DSS compliance attestations all provide assurance , confidence that the vendor's controls meet defined criteria based on auditor testing. Validation is direct confirmation that a specific control, in a specific configuration, prevents a specific risk in a specific context. The difference is between the general and the specific: assurance operates at the level of frameworks and criteria, while validation operates at the level of individual controls and specific attack scenarios.

The criteria gap is the mechanism through which assurance leaves specific risks unvalidated. Every assurance framework defines the criteria it assesses , the specific control requirements that must be met for the assurance opinion to be issued. Controls and risks that are not within the criteria framework are not assessed and not assured. The SOC 2 Trust Services Criteria cover broad categories of access management, availability, and confidentiality but do not specify testing of every possible authorisation bypass technique. A vendor whose SOC 2 confirms access controls operate effectively may have an authorisation model with specific vulnerabilities that the SOC 2 criteria were not designed to detect.

The application-specific risk gap is the most practical manifestation for TPRM. Every customer relationship creates specific risks based on the specific data, systems, and access patterns involved. A healthcare company's specific risk , cross-patient record access through API parameter manipulation , is a relationship-specific risk that a generic compliance framework will not specifically address. Validating relationship-specific risks requires testing that targets those specific risks in the specific context of the vendor relationship, not assurance from a framework that defines its own criteria independently of any specific customer's risk profile.

• Assurance framework criteria not covering specific customer risk
• Relationship-specific attack scenarios not in compliance criteria
• API authorisation bypass not a SOC 2 criterion
• Application-specific access control gaps missed by generic assurance
• Direct validation not performed for highest-risk customer-specific scenarios

Why this matters

Assurance versus validation matters for TPRM because the specific risks that customer relationships create , the particular data they expose, the particular access patterns they enable, the particular attack scenarios that are most relevant , are frequently not within the criteria of the compliance frameworks that provide the primary assurance evidence. The compliance assurance confirms the generic control programme. The customer-specific risk requires specific validation.

The regulated data dimension makes this particularly acute. Regulated data types , PHI in healthcare, PII under GDPR, cardholder data under PCI-DSS, financial data under financial regulations , create specific access control requirements that may not be fully represented in a vendor's general compliance framework criteria. A vendor whose SOC 2 confirms access controls operate may not have been specifically tested for the authorisation requirements that apply to the specific regulated data type the customer is sharing.

Where most teams get this wrong

The most consistent failure is accepting assurance as validation for the customer's specific risk. Assurance answers the framework's question. Validation answers the customer's question. Both questions may have different answers from the same vendor.

• Accepting assurance as validation for customer-specific risk
• Framework criteria treated as customer risk coverage
• No customer-specific risk validation for highest-risk scenarios
• Right-to-audit not used for specific validation testing
• Relationship-specific attack scenarios not tested

What good looks like

Mature assurance and validation programmes use compliance assurance as a foundation for understanding the general control environment and supplement it with specific validation testing for the highest-risk customer-specific scenarios , using penetration testing, API security testing, and targeted right-to-audit exercises to validate the specific controls the customer's risk profile requires.

• Assurance as foundation, validation as supplement , compliance provides general confidence, validation addresses specific risks
• Customer-specific risk scenarios identified , what specific attacks would most affect this relationship
• Targeted validation testing for highest-risk specific scenarios
• Right-to-audit exercised for validation , directed at customer-specific risk questions
• Validation findings feeding risk register alongside assurance-based assessment

Tooling

Application Security Testing , Burp Suite, OWASP ZAP, API security testing

Application-specific security testing tools enable validation of specific access control and authorisation vulnerabilities that compliance framework testing does not address. For TPRM practitioners, commissioning targeted API security testing focused on the specific access control risks in the vendor relationship provides the validation evidence that SOC 2 assurance cannot.

Penetration Testing , targeted right-to-audit exercises

Targeted penetration tests commissioned as right-to-audit exercises, focused on specific customer-defined risk scenarios, provide validation of the exact risks the customer relationship creates. For TPRM practitioners, developing customer-specific test scenarios for right-to-audit penetration tests enables the specific validation that generic compliance testing cannot produce.

Governance challenges

The governance challenge with assurance versus validation is the cost of specific validation testing. Targeted penetration tests and application security assessments require significant investment per vendor relationship. The governance resolution is risk-tiering the validation requirement , comprehensive specific validation for the highest-risk relationships where the customer-specific risk is most consequential, reliance on assurance for lower-risk relationships where the framework criteria are a sufficient proxy for the customer's risk.

• Identify the two or three customer-specific risk scenarios most material to each high-risk relationship
• Commission targeted validation testing for those specific scenarios
• Use right-to-audit for validation not just attestation
• Distinguish what SOC 2 has and has not tested for the specific vendor relationship
• Document the validation gap , which specific risks are covered by assurance and which require direct validation

If you are a small team

For your highest-risk vendor, write down the single attack scenario that would cause the most harm to your customers if it succeeded at that vendor , the specific access pattern, the specific data type, the specific exploitation path. Then ask whether the vendor's most recent SOC 2 report or ISO 27001 assessment specifically tested for that scenario. If the answer is no or unclear, you have identified the validation gap. That specific scenario is the scope for a targeted right-to-audit test that validates what the compliance assurance does not.

• Identify the single most harmful attack scenario for each high-risk vendor relationship
• Check whether compliance assurance specifically tested for that scenario
• Commission targeted validation testing for unvalidated high-priority scenarios
• Use right-to-audit for scenario-specific validation

What to require

Ask directly:

"We have identified [specific attack scenario, e.g. API parameter manipulation enabling cross-customer data access] as the highest-risk scenario for our relationship. Has your most recent SOC 2 or penetration test specifically tested for this scenario , and if not, would you consent to a targeted test of this specific scenario under our right-to-audit provision?"

Expect as evidence

• Confirmation of whether specific scenario was tested in existing assessments
• Consent to targeted validation testing for unvalidated scenarios
• Previous test results if scenario was assessed
• Remediation evidence if scenario was previously identified as a gap

A vendor who confirms SOC 2 access control assurance should be asked whether the specific access scenario most relevant to the customer's data has been specifically tested. Assurance covers the criteria. Validation covers the scenario.

How to evidence it

• Customer-specific risk scenario identification records
• Targeted validation testing records
• Validation gap documentation
• Right-to-audit exercise records for validation purposes

Key Takeaway

Assurance answers the framework's question. Validation answers the customer's question. The SOC 2 confirmed access controls operate , authentication, logging, reviews. The penetration test found that an authenticated user could access any patient's record through API parameter manipulation. Both findings are true. The assurance framework did not ask the customer's question. Only targeted validation testing did. Identify the customer-specific risk scenarios. Check whether existing assurance has tested them. Commission validation testing for the ones it has not. The framework provides the foundation. The customer's specific risk requires the supplement.

More on Compliance

• Audit Evidence Quality
• Audit Scope Limitations
• Compliance Automation Gaps