Adverse Scenarios and Accuracy Rates

This is part three of a three-part series about accuracy rates during identity verification. In part one, you’ll learn why pass rates can be misleading when comparing identity verification solutions. And in part two, you’ll learn the difference between accuracy rates during model development and deployment in the real world.

Imagine you're a bouncer. Every night, you grab a pair of dark sunglasses and your high-powered flashlight on your way out the door.

You work at a swanky lounge a few days each week and a college bar on the weekends. At both jobs, you check IDs to confirm the patrons are old enough to enter and make sure the ID they present actually belongs to them.

However, whether you’re at the nightclub or bar, each night is a little different. Stormy weather, guests with bad attitudes, worn-out IDs, or an overwhelming event can impact your ability to do your job well.

In the digital world, we call these challenging situations "adverse scenarios," and they're far more common than you might think. Just as a bouncer needs to make accurate decisions regardless of conditions, identity verification systems must maintain accuracy and security even when circumstances are less than ideal.

Examples of adverse scenarios during identity verification

One area where adverse scenarios often affect the identity verification process is when you’re collecting and processing images for government ID checks and selfie verifications.

Some of the adverse scenarios that may affect performance include:

Poor lighting conditions: Low or uneven lighting can make it difficult for facial recognition or document scanning systems to capture clear images.
Blurry images: Machines and manual reviewers may have trouble reading text or comparing facial features if the image is blurry.
Incorrect poses or angles: Some automated systems require users to position their images in a specific way to function accurately.
Device camera: Devices with lower-quality cameras can have trouble taking clear pictures.
Lamination glare or reflections: Light reflections on laminated documents can obscure key details.
Damaged or worn ID: Faded text, scratches, or torn edges can make it difficult to extract information.
Paper-like ID: Some countries issue IDs that resemble paper documents, which can lead to false matches in paper detection checks.
Document type: A model might not recognize or be able to assess government IDs from certain countries or regions.
Security features: Different government agencies may issue IDs with varying security features. Fewer and lower–quality security features can make distinguishing real and fake IDs more difficult.

How adverse scenarios affect identity verification metrics

Let’s go back to being the bouncer to understand how adverse scenarios affect commonly used metrics during identity verification. But first, a few quick definitions:

Pass rate: The percentage of users who have completed the verification process
Accuracy rate: How well the model correctly identifies legitimate users and denies illegitimate users. You can break this down into:
- True pass: Correctly verifying a legitimate user.
- False pass: Incorrectly verifying a bad actor as legitimate.
- True fail: Correctly denying an illegitimate user.
- False fail: Incorrectly denying a legitimate user.

We take a deep dive into pass rates in part one. Or, if you’re ready, put on your shades.

The lounge where you work primarily serves older customers. You quickly scan their IDs, but you’re mostly there to give them a sense of safety and exclusivity. In contrast, when you’re at the college bar, you scan the barcode and then closely check each ID to ensure it’s real and belongs to the person.

At the bar, your extra checks increase your accuracy and lead to a low pass rate because you do a great job of turning away underage students (true fails). Your boss approves the approach because they primarily worry about serving minors (false passes).

At the lounge, your accuracy is lower and your pass rate is higher because your quick checks miss a few fake IDs. However, most minors don’t want to sneak in and buy overpriced cocktails anyway. Your boss prefers accidentally letting in a few minors (false passes) to turning away older customers (false fails) or making customers wait too long.

Now, imagine the same adverse scenario happens at both venues. Perhaps a customer who looks like they’re in their early 20s has a scratched ID and you can’t scan its barcode. (In this case, they’re actually 20 and using a fake ID.)

At the college bar, you turn away the person to be safe, which keeps your accuracy high but lowers your pass rate. At the lounge, you let it slide because the rest of the ID checks out, resulting in a higher pass rate but lower accuracy rate.

The same challenge plays out differently because each environment has different risk tolerances and goals.

Part two of this series is all about accuracy, but there are two important takeaways here:

The types of checks you perform, the thresholds you enforce, and the specific scenarios you encounter can affect accuracy rates.
The same identity verification model will have different accuracy rates during production depending on the organization and use cases.

Have a plan for dealing with adverse scenarios

Let’s tie these points directly to verifying identities with a government ID and selfie.

If you accept images with glare over the ID portrait, your pass rate might increase, but your accuracy could drop. Automatically denying attempts with glare could thwart bad actors who try to use glare to trick the system, but it could also lead you to turn away legitimate users who have trouble taking a clear picture.

Instead of focusing on what to do when an image has glare, you can try to address the adverse scenario itself.

For example, give users clear instructions on capturing glare-free photos, let them try several times before failing the check, and offer an alternative verification option if they continue struggling.

Additionally, use other information, such as passive and active risk signals, to separate bad actors from legitimate users to increase your accuracy rate.

How Persona helps businesses handle adverse scenarios

Recognizing and mitigating adverse scenarios that degrade system performance is important for ensuring that identity verification, age assurance, and other processes are accurate, fair, and accessible.

Persona's verification platform is built with real-world challenges in mind. Here's how our features help you maintain accuracy, security, and compliance while ensuring legitimate users can verify their identity:

Smart capture technology and a guided user experience

Real-time feedback with automatic glare detection guides users in capturing higher-quality images.
AI-powered image enhancements and sophisticated optical character recognition (OCR) can improve less-than-perfect captures and read damaged or worn documents.
Automatic cropping and orientation correction improve image analysis.

Customizable configurations

Dynamic flows that can adjust verification requirements in real time based on risk levels.
Alternative and fallback verification methods when primary attempts face challenges.
No-code flow editor with customizable and brandable themes to improve conversions.

Dynamic routing based on risk

Passive, behavioral, and active fraud signals detect diverse fraud attempts.
Connections to authoritative and issuing databases improve identity assurance.
Link analysis uncovers fraud rings and large-scale coordinated attacks.

Manual review tools

Efficient queue management for cases that require human review.
Clear interface for reviewing multiple verification attempts.
Tools for communicating with users about verification issues.

Build identity verification processes for real-world conditions

The reality is that adverse scenarios will always exist. The key isn't to eliminate them but to handle them gracefully.

As a bouncer, your success came from having the right tools and experience to make educated decisions no matter the circumstances. That's exactly what Persona provides for digital identity verification: the tools and intelligence to verify identities accurately and securely, even when perfect isn't possible.

Ready to see how Persona can help your business handle adverse scenarios during identity verification? Contact us to learn more about building resilient verification flows that work in the real world.

What being a bouncer can teach us about adverse scenarios during identity verification (part 3)