Case study · Retail & eCommerce
An integration that fixes itself instead of paging someone
A home goods brand was running Shopify and its ERP on a nightly file that failed roughly weekly, always overnight. We replaced it with an event-driven integration that reconciles itself.
Engagement · Integration programme
Integration judged on what happens when it fails, not when it works.
- Industry
- Home goods & furniture
- Solution
- Shopify to ERP integration
- Platform
- Shopify
- Engagement model
- Dedicated integration team
- Scope
- Orders, stock, pricing, fulfilment, returns
- Pattern
- Event-driven with replay
- Constraint
- ERP remains the system of record
- Target
- No manual re-keying
Outcomes
What a reliable integration is worth
The old one worked most nights. Most is not a standard.
-
0
Orders re-keyed by hand
Orders flow from Shopify into the ERP as events, with failures queued and retried rather than dropped into somebody's morning.
By design: no manual order entry path remains
-
−98%
Integration incidents requiring a person
Transient failures retry automatically with backoff, and the position can be rebuilt by replaying the event log, so almost nothing needs a human.
Measured across integration incidents before and after
-
4 min
Stock accuracy window, from 24 hours
Stock changes in the ERP reach the storefront within minutes rather than overnight, which is the difference between an oversell and a sale.
Measured end to end under production load
Context
A nightly file with no error handling
It worked until it did not, and then somebody found out at nine in the morning.
The business
A home goods and furniture brand selling online and through a small number of stockists, running a mature ERP.
The starting point
Orders exported from Shopify overnight and imported into the ERP; stock exported back the same way. A failure meant a day of manual work.
The trigger
Overselling and delayed dispatch had become routine, and the operations team was spending mornings reconciling rather than shipping.
What they wanted
Near-real-time synchronisation, no manual re-keying, and an integration that recovers from failure without a person.
Constraints
The ERP stays the system of record and cannot be modified freely · the ERP has strict rate limits · orders must never be duplicated · failures must be recoverable rather than investigated.
System
What it runs at today
The integration as it runs today.
-
4 min
Stock window
Down from 24 hours
-
0
Manual re-keying
No path remains
-
−98%
Human incidents
Automatic recovery
-
Replay
Recovery model
Rebuild from the log
The engineering problem
Four failure modes of file-based integration
They are all the same failure: no state, no retry, no visibility.
-
A failed file is a lost day
When the whole day's orders move as one file, a failure loses all of them, and recovery is manual because nothing recorded what succeeded.
What we did
Per-order events with individual state, so a failure affects one order and retries by itself.
-
Stock a day out of date
Overnight stock export means the storefront sells against yesterday's position for the entire trading day.
What we did
Event-driven stock publication within minutes, respecting the ERP's rate limits rather than fighting them.
-
Duplicate orders on retry
Naive retries create duplicate orders in the ERP, which is worse than the original failure and takes days to untangle.
What we did
Idempotency keys on every write, so a retry is safe and a duplicate is impossible by construction.
-
Nobody knows what failed
File integrations fail silently. The first signal is usually a customer or a stock count.
What we did
Continuous reconciliation comparing both systems, with divergence surfaced immediately and the event log available for replay.
Architecture
How it fits together
Simplified — the shape of the system rather than every service in it.
-
Events
- Order events
- Stock events
- Fulfilment events
Every change published as a discrete, replayable event.
-
Queue & retry
- Backoff
- Dead letter
- Idempotency
Transient failures handled without a person; permanent ones isolated.
-
Rate limiting
- ERP limits
- Batching
- Prioritisation
The ERP protected from the integration rather than overwhelmed by it.
-
Reconciliation
- Continuous comparison
- Divergence alerts
- Replay
Correctness checked continuously, not at month end.
Idempotency is the load-bearing detail. Without it, automatic retry — which is the entire point — creates worse problems than it solves.
Solutions
What we built
An integration designed to be boring.
-
Event pipeline
Order, stock and fulfilment events.
-
Idempotent writes
Safe retry by construction.
-
Rate limiting
ERP protected during normal load and replay.
-
Stock sync
Positions published within minutes.
-
Fulfilment & returns
Dispatch, tracking and refunds both ways.
-
Reconciliation
Continuous comparison and divergence alerts.
Key capabilities
What it does day to day
Six capabilities in the integration.
| Capability | Runs | Refresh | What it does |
|---|---|---|---|
| Order sync | Automatic | Per order | Orders into the ERP as they are placed, idempotently |
| Stock sync | Automatic | 4-minute window | ERP stock positions published to the storefront |
| Pricing sync | Automatic | On change | Price changes from the ERP applied to the catalogue |
| Fulfilment | Automatic | Per shipment | Dispatch and tracking returned to Shopify and the customer |
| Returns | Automatic | Per return | Returns and refunds reflected in both systems |
| Reconciliation | Automatic | Continuous | Divergence detected immediately, with replay available |
Integrations
How the moving parts plug in
Small, independent, recoverable units of work.
Change happens
- Order placedIn Shopify
- Stock changesIn the ERP
- Event publishedImmediately
Queue
- Rate limitedTo ERP capacity
- Retried on failureWith backoff
- Idempotent writeNo duplicates
Both systems agree
- Applied
- ReconciledContinuously
- ReplayableFrom the log
Because reconciliation runs continuously rather than at period end, a divergence is a five-minute alert instead of a month-end investigation.
Security & data
What keeps the two systems honest
An integration that quietly drifts is more dangerous than one that loudly fails.
-
Idempotent writes
Every write carries a key, so retries cannot duplicate.
-
Event replay
The integration state can be rebuilt from the log rather than reconstructed by hand.
-
Continuous reconciliation
Both systems compared continuously, with divergence surfaced immediately.
-
Rate protection
The ERP is never overwhelmed, even during a replay or a catch-up.
The brief
Design the integration around its failures
The brand asked for a faster sync. Frequency was not the real problem — the old integration had no concept of per-record state, so any failure was total and any recovery was manual.
We designed for the failure case first: small events, idempotent writes, automatic retry and continuous reconciliation. The speed improvement came along for free.
- Per-order events rather than daily files
- Idempotency keys on every write
- Automatic retry with backoff and dead-lettering
- Continuous reconciliation with replay
What the integration had to respect
- 01An ERP that is the system of record and cannot change freely
- 02Strict ERP rate limits
- 03Orders that must never duplicate
- 04Failures that must recover without a person
Process
We ran it in parallel until it was dull
The only honest test of an integration is running both and comparing every record.
-
Failure analysis
Six months of integration failures categorised, which showed the problem was recovery rather than frequency.
-
Event pipeline
Built alongside the nightly file, writing to a shadow environment and compared record by record.
-
Idempotency
Proven by deliberately replaying production event volumes and confirming no duplicates.
-
Cutover by domain
Stock first, then orders, then fulfilment and returns, each with the file still running behind it.
-
File retirement
The nightly export switched off only after a full peak trading period with zero divergence.
Technology
Event-driven integration between Shopify and the ERP
Nothing exotic — the value is in the failure handling.
Commerce
- Shopify
- Admin API
- Webhooks
Integration
- Event queue
- Retry & backoff
- Dead letter queue
Correctness
- Idempotency keys
- Continuous reconciliation
- Event replay
Operations
- Divergence alerting
- Throughput monitoring
- Replay tooling
Business impact
What changed for operations
Three outcomes the operations team felt immediately.
-
Mornings returned to shipping
The daily reconciliation routine no longer exists.
-
Overselling largely ended
Stock accurate within minutes rather than a day.
-
Failures stopped needing people
Incidents requiring intervention down 98%.
The result
The integration stopped being something anyone thinks about
Stock is accurate within four minutes instead of twenty-four hours, no order is ever re-keyed, and incidents requiring a human fell 98%.
The operations team's morning reconciliation routine no longer exists, which was worth more to them than the speed.
- Zero manual order re-keying
- Stock accurate within 4 minutes
- Human-intervention incidents down 98%
- Recovery by replay rather than by hand
What we hold to in integration work
- 01Design for the failure case first
- 02Idempotency is what makes automatic retry safe
- 03Reconcile continuously, not at period end
- 04Run in parallel until divergence is zero
Free discovery call
Have an idea? Let's turn it into AI-powered software.
Book a free discovery call with our experts. Share your idea and we will help you shape the scope, timeline and budget, under NDA.
- Free consultation
- NDA before we talk
- Transparent estimate