Rethink Identity Verification Pass Rates

This is part one of a three-part series about accuracy rates during identity verification. In part two, you’ll learn the difference between accuracy rates during model development and deployment in the real world. And in part three, you’ll discover what being a bouncer can teach you about adverse scenarios during identity verification.

Identity providers often advertise their high pass rate as a compelling selling point. They might claim that you can use their platform to approve up to 98% of new users automatically. But a generic, high-level pass rate probably isn’t telling you what you actually need to know.

Pass rates fluctuate over time. The same tool can have different pass rates depending on the industry, geography, and verification method. Most importantly, pass rates don’t tell you if you should allow those users to pass through identity verification.

What is a pass rate in identity verification?

The pass rate of an identity verification model refers to the percentage of users who attempt and successfully complete the verification process. Companies generally want high pass rates because they want users to easily go through an onboarding or reverification process.

But pass rates aren’t the same as accuracy rates.

A high pass rate might indicate that the verification system does a great job of guiding people through the process and handling one-off challenges. Or, the model might be so lenient that it lets everyone through, including unqualified and fraudulent users.

How do you define and break down pass rates?

Try to align on a few common definitions to bring clarity to your discussions about pass rates and identity verification metrics. Such as:

Pass rate: The percentage of users who have completed the verification process
Accuracy rate: How well the model correctly identifies legitimate users and denies illegitimate users. You can break this down into:
- True pass: Correctly verifying a legitimate user.
- False pass: Incorrectly verifying a bad actor as legitimate.
- True fail: Correctly denying an illegitimate user.
- False fail: Incorrectly denying a legitimate user.

People also often use the terms “false positives” and “false negatives,” but the terms above are more precise because the definition of a “positive” can differ based on the context.

For example, in identity verification, a false positive refers to incorrectly verifying a bad actor as good. But in fraud detection, a false positive refers to incorrectly flagging a good account or transaction as fraudulent.

4 things that can affect your pass rates

Focusing solely on pass rates is often a mistake because pass rates include true passes and false passes. You likely want to understand the breakdown when you’re evaluating an identity verification model. But that’s not the only shortcoming. The same identity verification tool often has different true pass and false pass rates depending on the circumstances.

Four common factors that can affect pass rates include:

1. Your user population

A completely accurate identity verification system will result in a pass rate that reflects the risk associated with your underlying population.

In other words, if you’re in a high-risk industry and 20% of the people who try to verify their identities are bad actors, you’d want your pass rate to be 80%. A higher pass rate is actually a bad sign because it indicates there are false passes.

The sophistication of the fraudsters attacking your systems can also be relevant. For example, a verification system might easily detect preteens trying to trick their way onto an online gaming platform. But the same system might struggle when facing sophisticated fraudsters trying to break into a fintech.

2. The verification methods

Pass rates can also vary depending on the verification type, such as government ID, selfie, or document verification. In some cases, this coincides with the impact of different populations’ norms.

For example, when you rely on database verifications, you may have trouble verifying users who are underbanked, don’t use credit, or are new to the country. If you target people from those populations, you might expect a low pass rate, and you should probably look into alternative identification methods.

Sometimes, a verification tool’s limitations can also come into play. Say you’re expanding into a new region where people commonly have three identification documents, but the tool only supports two. Your pass rate might be lower because you can’t verify people who want to use that third identification document.

3. Your configurations and preferences

Two organizations with identical populations, risk levels, verification methods, and verification tools could still have different pass rates because of their preferences.

An organization prioritizing growth above all else might accept lower risk thresholds or require fewer checks, which could lead to a higher pass rate. Another might take a more cautious approach that increases accuracy but results in a lower pass rate.

4. Adverse scenarios

Adverse scenarios are the real-world situations that can negatively affect pass and accuracy rates. For instance, poor lighting, blurry images, and odd angles can lead to lower pass rates for government ID or selfie verifications. We explore adverse scenarios in depth in part three of this series.

Optimize your pass rates with Persona

A high pass rate might look good on paper, but it can also be misleading because it leaves out the nuance: your users, your verifications, your risk appetite, and your configurations. In other words, your context.

At Persona, our identity platform helps you build customized and automated verification flows that you can adapt as your needs change. Whether you’re optimizing for growth, safety, or a little of both, you can find the proper balance and aim for pass and accuracy rates that align with your goals.

Stay tuned for part two, where we’ll explore the gap between identity verification model accuracy during production and deployment. Or, contact us to see how Persona can help you automate onboarding without letting bad actors through.

Identity verification pass rate metrics aren’t telling you the whole truth (part 1)