Identity Anomaly Detection Gaps

A financial services vendor's identity threat detection system had generated zero alerts during the twenty-seven minutes that an attacker operated in their environment following a successful credential stuffing attack on a finance analyst's account. Post-incident analysis found that the attacker's behaviour contained multiple individually anomalous signals , authentication at 02:14 from a Nigerian IP, sequential access to seventeen applications within forty-two minutes, download of six hundred files from the document management system, and a logout followed immediately by a second login from a different IP. None of these signals had generated an alert. The geographic detection was configured to alert on access from a list of blocked countries , Nigeria was not on the blocked list. The application access count detection threshold was twenty applications per hour , seventeen was below it. The file download detection threshold was one thousand files per session , six hundred was below it. The consecutive login detection was configured to alert on concurrent sessions, not sequential logins from different IPs. Every individual signal was below its threshold or in the wrong detection category. A security analyst who reviewed the combined access pattern during the post-incident analysis identified it as an obviously anomalous event within thirty seconds of seeing the aggregated timeline.

What are Identity Anomaly Detection Gaps, Really?

Identity anomaly detection is the capability to identify access patterns that deviate from established normal behaviour , detecting authentication events, access patterns, data transfers, and session characteristics that are inconsistent with a user's baseline behaviour or that represent known attack patterns. Effective anomaly detection requires two complementary capabilities: threshold-based rules that detect specific high-risk events above defined thresholds, and behavioural analytics that detect patterns that are anomalous relative to a user's established baseline even when no individual threshold is exceeded.

The threshold-only detection gap is the structural limitation that the hook scenario illustrates. Rule-based detection systems define specific thresholds for specific signals , if X exceeds Y, generate alert. These rules are effective for attacks that produce signals above their respective thresholds. They miss attacks where every individual signal is below its threshold but the combination of signals is clearly anomalous. The attacker who deliberately stays below every individual threshold , downloading 600 files rather than 1,000, accessing 17 applications rather than 20, authenticating from a country that is not on the blocked list , has calibrated their behaviour to the detection system's published thresholds. Every threshold rule has an evasion strategy. Combined behavioural analytics are significantly harder to evade because the baseline is the individual user's normal pattern, not a universally published threshold.

The baseline staleness problem is a secondary anomaly detection gap. Behavioural analytics establish baselines of normal user activity , typical working hours, typical access patterns, typical application usage. Baselines that were established for an employee who worked primarily in the London office become inaccurate when that employee transitions to a remote work model, takes an international assignment, or changes function. An anomaly detection system operating on a stale baseline will generate false positives for legitimate access changes and miss genuine anomalies that are within the (now inaccurate) baseline's expected range. Baseline maintenance , continuously updating baselines to reflect current normal behaviour , is as important as baseline establishment.

• Threshold-only detection , individual signal thresholds without combined behavioural pattern analysis
• Below-threshold attack evasion , attacker behaviour calibrated to stay below individual detection thresholds
• Stale baselines , behavioural baselines not updated to reflect current normal behaviour
• Detection category misalignment , anomalous behaviour in a detection category that the rule set does not cover
• No combined signal correlation , individual signals evaluated independently rather than as a combined access profile

Why this matters

Identity anomaly detection matters for TPRM because vendor staff accessing customer environments create an identity behaviour pattern that should be detectable as anomalous when it deviates from normal support or operational activity. An attacker who has compromised a vendor support engineer's credential and is accessing customer environments at unusual times, in unusual patterns, or with unusual data access behaviour should be detectable by a well-configured anomaly detection system. The detection gaps described in the hook scenario represent the real-world performance of detection systems that rely on threshold rules without behavioural analytics.

The vendor access detection challenge is the baseline quality problem. Vendor support access to customer environments is often infrequent and variable , support engineers connect when customers have issues, at times determined by incident schedules rather than regular patterns. Establishing a reliable behavioural baseline for a support engineer who connects three times per month in response to incidents is harder than establishing a baseline for an employee who accesses the same systems daily at consistent times. The lower baseline quality for vendor access creates higher false positive rates and lower anomaly detection effectiveness for precisely the identity population where detection matters most.

Where most teams get this wrong

The most consistent failure is evaluating detection capability based on the list of configured rules rather than the performance of those rules against realistic attack scenarios. A comprehensive list of threshold rules describes the detection architecture. Testing those rules against the actual attack scenarios they are designed to detect , including below-threshold attacks that require combined signal analysis , describes the detection performance. The two are often significantly different.

• Evaluating detection by rule list rather than performance against attack scenarios
• No combined signal correlation , signals evaluated independently
• Threshold calibration not tested against realistic attacks
• Behavioural baselines not maintained for current activity patterns
• Vendor access baselines not established , infrequent vendor access not baselined

What good looks like

Mature identity anomaly detection programmes combine threshold rules for known high-risk signals with behavioural analytics that evaluate combined access profiles against individual baselines , using machine learning to identify patterns that deviate from established normal behaviour even when no individual threshold is exceeded.

• UEBA with individual behavioural baselines , each user's normal pattern baselined and deviations detected
• Combined signal correlation , multiple sub-threshold signals evaluated as a combined profile
• Baseline maintenance , baselines continuously updated to reflect current normal behaviour
• Attack scenario testing , detection performance validated against realistic attack patterns including below-threshold scenarios
• Vendor access baseline establishment , specific baselines for vendor and support account access patterns

Tooling

UEBA , Exabeam, Microsoft Sentinel UEBA, Varonis

User and Entity Behaviour Analytics platforms establish individual behavioural baselines and detect deviations through machine learning , evaluating access patterns holistically rather than as independent threshold checks. Exabeam specifically uses a timeline-based approach that correlates multiple signals across a user session into a combined anomaly score. For TPRM practitioners, asking whether the vendor uses UEBA with individual behavioural baselines in addition to threshold-based detection provides the specific combined analysis capability question.

Identity Threat Detection , Microsoft Entra ID Protection

Microsoft Entra ID Protection provides risk scoring that combines multiple identity signals , sign-in risk, user risk, impossible travel, leaked credentials, and behavioural anomalies , into a composite risk score that can trigger conditional access challenges or block access. The composite scoring approach addresses the below-threshold individual signal problem by evaluating signals in combination. For TPRM practitioners, asking whether the vendor uses composite risk scoring for authentication decisions provides a combined signal analysis question.

Governance challenges

The governance challenge with identity anomaly detection is the alert fatigue versus missed detection trade-off. Sensitive detection thresholds and low baseline deviation tolerances generate frequent alerts, many of which are false positives. Security teams that face alert fatigue begin to dismiss alerts without investigation , reducing the effective detection rate for genuine attacks. The calibration challenge is setting thresholds and deviation tolerances that catch genuine attacks while producing a manageable false positive rate that does not create the alert fatigue that causes genuine alerts to be missed.

• Ask what detection capability would catch the hook scenario , 2am access, multiple app access, below-threshold download
• Ask whether UEBA is deployed in addition to threshold rules
• Ask whether detection was tested against realistic below-threshold attack scenarios
• Ask about vendor access baseline establishment , whether infrequent vendor access is baselined
• Include anomaly detection performance in vendor security assessment , not just capability confirmation

If you are a small team

Present the hook scenario to your highest-risk vendor's security team: at 2am, one of your support engineers' accounts authenticated from Lagos, accessed seventeen applications in forty minutes, and downloaded six hundred files before logging out. Would that have generated an alert? Walk through their detection architecture against that specific scenario. If the answer is no alert because Nigeria is not blocked and the download count is below threshold, their detection relies on threshold rules without combined behavioural analysis. That conversation reveals detection performance better than any questionnaire.

• Present the hook scenario and ask whether it would have generated an alert
• Ask whether UEBA is deployed with individual behavioural baselines
• Ask whether combined signal correlation exists or signals are evaluated independently
• Ask whether vendor/support account access patterns are specifically baselined

What to require

Ask directly:

"If one of your support engineers authenticated at 2am from Lagos, accessed seventeen applications in forty minutes, and downloaded six hundred files , would that have generated an identity anomaly alert in your detection system, and which specific detection rule would have fired?"

"Does your identity anomaly detection use behavioural analytics with individual user baselines to detect combined access profile anomalies , or does it rely primarily on threshold-based rules for individual signals?"

Expect as evidence

• Attack scenario detection analysis , which signals the described attack would trigger
• UEBA deployment with individual behavioural baselines
• Combined signal correlation capability
• Vendor access baseline establishment process

A vendor whose detection would not have alerted on the hook scenario has confirmed the threshold-only gap. A vendor who can describe exactly which combined signals would have generated an alert and at what point in the session has confirmed behavioural analytics capability. The scenario is the test. Apply it.

How to evidence it

• Detection scenario testing records
• UEBA deployment confirmation with baseline documentation
• Combined signal correlation capability
• Vendor access baseline records

Key Takeaway

The attack was obvious to a human who looked at the combined timeline. The detection system never saw the combined timeline , it evaluated each signal independently against its threshold. Two in the morning was not a blocked hour. Lagos was not a blocked country. Seventeen applications was below the application threshold. Six hundred files was below the download threshold. Every signal passed individually. The combined profile failed obviously. Threshold-based detection catches attacks that exceed their thresholds. Behavioural analytics catches attacks whose combined profile is anomalous even when every individual signal is below threshold. The attacker who knows the thresholds stays below them. They cannot stay below a behavioural baseline they have never seen.

More on Security

• API Identity Misuse
• Access Certification Effectiveness
• Access Policy Misconfigurations