Model Theft Risks

A specialised AI vendor had developed a proprietary natural language understanding model that significantly outperformed competing models on their target use case , legal contract analysis. The model had been developed over three years, trained on a large proprietary corpus of annotated legal documents, and refined through extensive collaboration with legal professionals. The model represented the vendor's primary competitive moat. The vendor exposed the model through a well-secured API: mutual TLS authentication, API key management with fine-grained scoping, rate limiting, and anomaly detection for unusual query volumes. The security architecture was genuine and well-implemented. The vendor's forty enterprise customers integrated the model into their own legal technology products , document review tools, contract management platforms, and legal research applications. Each integration created a pathway from the vendor's model through the enterprise customer's application. A threat actor targeting the vendor's model identified that one of the forty enterprise customers , a startup with limited security resources , had implemented the API integration with minimal security controls: the API key was hardcoded in the application's source code, which was partially visible in a GitHub repository. The attacker obtained the API key from the GitHub repository and used it to systematically query the vendor's model API over thirty days , making four hundred thousand queries across the legal contract analysis domain. The vendor's anomaly detection flagged the unusual query volume on day twenty-eight. By then, the attacker had accumulated sufficient query-response pairs to train a surrogate model that approximated the vendor's contract analysis capabilities. The vendor had secured their model. The model was stolen through a customer's insecure integration.

What is Model Theft Risk, Really?

Model theft risk is the risk that a proprietary AI model's decision logic and capabilities can be reconstructed by an attacker through systematic querying of the model's API , either directly or through a third party with legitimate API access , without the attacker ever accessing the model's weights, architecture, or training data. Model theft differs from model extraction by the motivation: extraction is typically for competitive intelligence or competitive advantage, while theft describes the act of appropriating the model's capabilities without authorisation. The two are technically similar , both use systematic API querying to train a surrogate model , but they may have different legal implications.

The extended attack surface problem is the supply chain dimension. An AI vendor who exposes their model through an API to enterprise customers creates an extended attack surface: every enterprise customer who integrates the API creates a pathway to the model. The vendor's own API security may be excellent. The enterprise customer who implements the integration with a hardcoded API key, inadequate credential management, or insufficient access controls creates a weakly secured pathway to the vendor's model. An attacker targeting the model need not penetrate the vendor's security , they need only find the least-secure enterprise customer integration and exploit that.

The surrogate model quality dimension determines the theft consequence. Simple models with limited output complexity , binary classifiers, regression models with few features , can be approximated with relatively few queries. Complex models with rich outputs , large language models, multi-class classifiers with detailed outputs, models with fine-grained confidence scores , require more queries but can still be substantially approximated given sufficient time and query volume. The value of the stolen model depends on the quality of the surrogate , a high-quality surrogate that closely approximates the original's capabilities on the target use case is effectively the original's competitive advantage transferred to the attacker.

The intellectual property dimension is the legal framework for model theft. Proprietary AI models may be protectable as trade secrets , the specific model architecture, training methodology, and trained weights may constitute trade secret information. Model theft through API querying , using the model's outputs to train a surrogate , may constitute trade secret misappropriation even when the API access that enabled it was obtained through legitimate means (as when the attacker uses a stolen API key from an enterprise customer). The legal framework for AI model theft is still developing, but the trade secret framework is increasingly applicable.

Why this matters

Model theft matters for TPRM from two directions. First, if the enterprise deploys a vendor's proprietary model and that model is stolen through the enterprise's insecure integration, the enterprise may be liable for the theft , as the source of the leaked API key. Second, if the enterprise has developed proprietary AI models that it shares with partners or customers, each integration point is a model theft risk surface that must be governed.

Where most teams get this wrong

The most consistent failure is assessing model security at the vendor's API level without assessing the security of the enterprise's own integration implementation. The vendor's model security is only as strong as the weakest integration through which it is accessed.

• Vendor API security assessed without enterprise integration security
• API key management in enterprise application not assessed
• Extended API surface through customer integrations not considered
• Surrogate model feasibility not assessed for exposed model
• Model theft legal framework , trade secret protections not assessed

What good looks like

Mature model theft prevention programmes implement API key management best practices in all model integrations , no hardcoded credentials, key rotation, key scoping to minimum necessary endpoints , combined with query volume monitoring at the integration level to detect systematic querying consistent with extraction attempts.

• API key management in integration , no hardcoding, rotation, minimum scope
• Query volume monitoring per API key , baseline and anomaly detection
• Vendor-side extraction detection , surrogate model threat modelled
• Integration security requirements , vendor requires security controls for enterprise customers
• Trade secret protection framework for proprietary models

Tooling

API Security , API gateway rate limiting, per-key anomaly detection, credential scanning in CI/CD

Credential scanning tools , GitHub's secret scanning, Gitleaks, Truffelhog , detect API keys and credentials accidentally committed to source code repositories. For TPRM practitioners asking a vendor: do you require credential scanning in your enterprise customers' development pipelines as a condition of API access? , this question reveals whether the vendor is protecting against the extended attack surface their customer integrations create.

Governance challenges

The governance challenge with model theft through customer integrations is the vendor's limited control over enterprise customer implementation security. The vendor can require security controls contractually, but enforcement is difficult. The governance resolution is technical enforcement at the API level , API key scoping that limits query volume and endpoint access, anomaly detection that identifies extraction-consistent query patterns regardless of which API key is used.

• Require API key management best practices in integration contracts
• Implement per-key query volume monitoring for extraction detection
• Conduct credential scanning in CI/CD pipeline for model API keys
• Assess surrogate model feasibility for proprietary models
• Include model theft protections in API terms of service

If you are a small team

For any model you expose through an API , vendor or customer , implement two controls. First, audit all integration implementations for hardcoded API keys , use a credential scanner on all code repositories that implement the integration. Second, implement per-API-key query volume monitoring with anomaly alerting , any key that exceeds a threshold volume of systematic queries should trigger investigation. Those two controls address the most common model theft pathways: credential exposure and volume-based extraction.

• Audit all integration code for hardcoded API keys using credential scanner
• Implement per-API-key query volume monitoring with anomaly alerting
• Rotate any API keys found in code repositories immediately
• Assess surrogate model feasibility for proprietary models

What to require

Ask directly:

"Do you require credential scanning and API key management best practices from enterprise customers who integrate your model API , and do you have per-key query volume monitoring that would detect systematic extraction attempts from any of your customer integrations?"

Expect as evidence

• Enterprise customer API security requirements
• Per-key query volume monitoring confirmation
• Extraction detection capability
• Surrogate model threat assessment

A vendor who confirms API security should be asked about the security of enterprise customer integrations. The vendor's API controls the direct access. The customer integrations are the extended surface through which the model can be accessed indirectly.

How to evidence it

• API key management in integration documentation
• Credential scanning implementation
• Per-key volume monitoring records
• Model theft trade secret protection framework

Key Takeaway

Vendor API: mutual TLS, key management, rate limiting, anomaly detection. Enterprise customer GitHub: hardcoded API key. Attacker: API key obtained from GitHub. Four hundred thousand queries. Day twenty-eight: anomaly detection fires. Surrogate model: trained. The vendor's model was secured. The model was stolen through a customer's insecure integration. The vendor's API security is a direct attack control. The extended surface through customer integrations is the indirect attack pathway the vendor's security cannot close alone. Per-key query monitoring detects the extraction regardless of which key is used. Credential scanning in customer pipelines prevents the API key from reaching GitHub. Both are vendor obligations extended to customer integration security requirements.

More on AI governance

• AI API Key Exposure
• AI Adversarial Attacks
• AI Attack Surface Expansion