Case study · Payments

Catch unusual transactions without blocking good customers

Fraud rules catch known patterns and block too many genuine customers. This is how we added machine learning risk scores to transaction processing, send uncertain cases to analysts and measure both fraud caught and customers wrongly stopped.

Start a project like this See more of our work

Project overview

Real-time risk scoring alongside existing rules.

Industry: Payments & lending
Client: Confidential
Solution: Anomaly and fraud detection
Platform: Azure ML + API
Timeline: 12–18 weeks
Team: 3–4 people
Latency: Real-time scoring
Review: Analyst queue
Governance: Model documentation and monitoring

Expected business outcomes

What the client set out to achieve

Measured on historical data first, then in shadow mode on live traffic.

20–40% Fewer false positives at the same catch rate
Fewer genuine customers stopped for the same fraud detected.

Target range from shadow-mode comparison with current rules
Real time Scoring at transaction time
A risk score returned within the payment flow.

Latency budget agreed with engineering
Documented Model decisions
Features, thresholds and performance recorded for audit.

Model governance pack

Business challenges

Where the client started

Rules work, but they're blunt.

Rules

Dozens of rules added after each incident.

False positives

Genuine customers blocked and calling support.

Analysts

Review queues full of low-risk cases.

New patterns

Fraudsters adapt faster than rules are written.

Governance

Regulators expect decisions to be explainable.

Proposed solution

What the build covers

Machine learning alongside rules, with humans reviewing grey areas.

Feature engineering

Behavioural features such as velocity, device and history.
Risk models

Supervised and anomaly models scoring each transaction.
Scoring API

Real-time scoring in the transaction flow with fallbacks.
Analyst queue

Uncertain cases reviewed with reasons shown.
Shadow mode

Models run silently alongside rules before they decide anything.
Monitoring

Catch rate, false positives and drift tracked.

Project requirements

What the client asked us for

The request was to catch more fraud. Equally important is stopping fewer good customers, so both are measured from the start.

Models run in shadow mode first, compared with existing rules on live traffic, before they influence any decision.

What the project had to deliver

01Risk scores in real time
02Fewer genuine customers blocked
03Analysts on the right cases
04Explainable decisions

Key challenges & considerations

What made it harder, and what we did

Few confirmed fraud labels

What we did

Anomaly detection and analyst feedback were used while confirmed fraud labels were still scarce.
Strict latency requirements

What we did

The scoring service was built and tested to the latency budget.
Regulatory explainability

What we did

Every alert carried its reasons, and the models were documented for audit.
Adversaries changing behaviour

What we did

Models ran in shadow mode first and were retrained as patterns changed.

What we watched for

Measure false positives, not just fraud caught
Always run in shadow mode first
Document models for audit
Retrain as patterns change

Development process

Shadow first, then decide

No customer is affected until the model has proved itself.

Stage 1
Data and labels

Transactions and confirmed outcomes prepared.
Stage 2
Models

Models built and validated on historical data.
Stage 3
Shadow mode

Live scoring alongside rules without affecting decisions.
Stage 4
Decision rollout

Thresholds set with fraud and product teams.
Stage 5
Monitoring

Ongoing tracking and periodic retraining.

Technology stack

Real-time ML scoring

Low-latency scoring with full audit trails.

Machine learning

Python
scikit-learn / LightGBM
Azure Machine Learning

Data

Microsoft Fabric or Azure SQL
Feature pipelines

Serving

Batch scoring
REST API
ERP/CRM integration

Monitoring

Accuracy tracking
Drift alerts
Retraining schedule

Quality assurance & deployment

Measured against a test set before launch

AI quality was measured, not judged from a demo. A test set of real examples with agreed answers decided whether the system was ready, and the same set was re-run after every change.

A test set of real examples with agreed answers
Accuracy thresholds agreed with the business before launch
Safety checks: refusals, permissions and prompt-injection tests
A person reviewing low-confidence output
Cost and response time measured at real volumes

Deployment

01Decision rolloutThresholds set with fraud and product teams.
02MonitoringOngoing tracking and periodic retraining.
03Pilot, then staged rolloutA small group of real users first, with results reviewed weekly before widening access.
04Monitoring and warrantyAccuracy, drift, usage and cost tracked after launch, and six months of free bug fixing.

Final deliverables

Sharper decisions, fewer blocked customers

Transactions are scored in real time, clear cases are handled automatically and analysts focus on genuinely uncertain ones.

Every decision is documented and performance is monitored continuously.

Real-time risk scores
Shadow-tested models
Focused analyst queues
Audit-ready governance

Talk about a project like this

Handed over to the client

01Production AI serviceDeployed in the client's own cloud tenant
02Source code and modelsIn a repository the client can access
03Test set and resultsThe evidence behind the launch decision
04MonitoringAccuracy, usage and cost
05Six-month warrantyFree bug fixing after delivery

Success metrics

How success is measured

Measure	Outcome	How it is measured
Fewer false positives at the same catch rate	20–40%	Target range from shadow-mode comparison with current rules
Scoring at transaction time	Real time	Latency budget agreed with engineering
Model decisions	Documented	Model governance pack

Verified reviews

What clients say about our custom software work

Verified reviews from our clients on the same kind of work, published on Clutch.

5.0 27 verified reviews

5.0 Custom Software Development

Web-Based Coupon App Development

The work produced by the team was of exceptional quality.

Jon Thies CTO, Model Rocket Feb 2022 – Mar 2023
Verified review on Clutch (Jon Thies, opens in a new tab)
5.0 Custom Software Development

Mobile & Web Platform Development & Design

Silver Scintilla was responsive, respectful, and enjoyable to work with, quickly addressing any questions or requests.

DeShawn Brown CEO & Founder, Lithios Jun 2021 – Nov 2022
Verified review on Clutch (DeShawn Brown, opens in a new tab)
5.0 Custom Software Development

Web-Based HRM System Development

Silver Scintilla provided the best solution with scalable functionality, personalized features, and robust security.

Suheb Khan Director, Collaborative Insight Technologies Jul 2020 – Jun 2022
Verified review on Clutch (Suheb Khan, opens in a new tab)

See All Testimonials Read All Reviews on Clutch

Catch unusual transactions without blocking good customers

What the client set out to achieve

Where the client started

What the build covers

Feature engineering

Risk models

Scoring API

Analyst queue

Shadow mode

Monitoring

What the client asked us for

What made it harder, and what we did

Few confirmed fraud labels

Strict latency requirements

Regulatory explainability

Adversaries changing behaviour

Shadow first, then decide

Data and labels

Models

Shadow mode

Decision rollout

Monitoring

Real-time ML scoring

Machine learning

Data

Serving

Monitoring

Measured against a test set before launch

Sharper decisions, fewer blocked customers

How success is measured

What clients say about our custom software work

Services we used on this project

AI & ML development

AI proof of concept

Payment app development

Have an idea? Let's turn it into AI-powered software.