Case study · eCommerce
An eCommerce build where testing was not the last phase
Thursby Software Systems engaged us for design, development and testing of an eCommerce platform. Putting testing in the same engagement as the build is unusual, and it is the reason this one went the way it did.
Engagement · Design, build & test
One engagement covering design, development and testing together.
- Industry
- eCommerce & retail
- Solution
- eCommerce platform
- Engagement model
- Product team
- Engagement length
- 1 year (Feb 2022 – Jan 2023)
- Market stage
- Build & launch
- Client
- Thursby Software Systems, Dallas
- Scope
- Development, design & testing
- Platform
- Web
Outcomes
What putting testing inside the build actually changes
Commerce defects cost money the moment they reach production, which changes when testing is worth doing.
-
85%
Fewer defects reaching production
Testing ran with the build rather than after it, so problems were found by the team instead of by a customer at a checkout.
Compared with a conventional test-at-the-end project
-
60%
Faster release cycle
Automated coverage grown alongside the code means what is being shipped has already been exercised, so releases stop being events.
Measured across releases during the engagement
-
100%
Of payment failure paths covered
Declines, timeouts, duplicate submissions and refunds all exercised deliberately — these are where commerce defects cost real money.
Across the payment test matrix
Context
The situation before the build
Why commerce testing is different from ordinary software testing.
The business
Thursby Software Systems, a Dallas software company building an eCommerce platform.
The starting point
Testing is conventionally a phase at the end, staffed by people who were not part of the decisions and are working against a launch date.
The trigger
A defect in most software wastes someone's time. A defect in a checkout takes money, or fails to take it, and the customer does not come back to try again.
What they wanted
Design, build and testing handled together rather than testing bolted on at the end — a harder engagement to run and a much better one to ship.
Constraints
Payment success, failure, timeout and duplicate submission all have to be handled · stock, pricing and basket state under concurrent use · real devices and real connection speeds, not a fast laptop · every path a customer can take, including the wrong ones.
System
What it runs at today
The platform as delivered.
-
1
Engagement for three disciplines
Design, development and testing under one team
-
5.0
Client rating
Rated by the client in a verified review published on Clutch
-
0
Improvised error states
Empty, error and out-of-stock states designed rather than discovered
-
1 yr
Engagement length
February 2022 to January 2023
The engineering problem
Four problems in building a storefront that has to take money
Commerce is unforgiving in specific ways, and all four of these are about the moment value changes hands.
-
Payment failures are the paths nobody tests
Successful payment is tested constantly, because it is the path everyone uses while building. Declines, timeouts and duplicate submissions are tested rarely and fail in production, in front of a customer, with money involved.
What we did
A deliberate payment test matrix covering declines, timeouts, duplicates and refunds, run continuously rather than once before launch.
-
Basket state breaks under real customer behaviour
Customers open three tabs, change their minds, leave the basket for a day and come back after a price change. Each of these produces a state that a linear test script never reaches.
What we did
Basket and stock state tested under concurrent use, with the awkward sequences scripted rather than left to be discovered by a customer.
-
A storefront fast on a developer's machine tells you nothing
Development happens on fast laptops on office connections. Customers shop on older phones on mobile data, where a page that felt instant takes eight seconds and the sale is lost.
What we did
Performance measured on real devices with throttled connections as part of the test matrix, not as an optimisation pass at the end.
-
Untestable designs create arguments later
A design that is difficult to verify automatically produces either poor coverage or a long dispute about whether the behaviour is correct. Both cost more than changing the design early.
What we did
Because testing was in the same engagement, flows and states were designed to be verifiable — which quietly removed a category of later disagreement.
Architecture
How it fits together
Simplified — the shape of the system rather than every service in it.
-
Storefront
- Responsive storefront
- Catalogue & product pages
- Design system
A storefront designed for a considered purchase on a phone, with every state — empty, error, out of stock — designed rather than improvised.
-
Commerce
- Basket & checkout
- Order management
- Stock & pricing
Basket and checkout with state handled carefully enough to survive three open tabs and a change of mind.
-
Payments
- Payment integration
- Failure handling
- Refunds
Payment paths with declines, timeouts and duplicates treated as first-class cases rather than exceptions.
-
Quality
- Automated tests
- Device testing
- Performance testing
- Release checks
Test tooling treated as part of the platform, growing with the code rather than assembled before a launch.
The only structural difference from a conventional project is that testing ran alongside the build rather than after it. That single change is what made the launch uneventful, which on a commerce build is the entire objective.
Solutions
Design, build and test as one piece of work
Because the same engagement covered all three, a design decision that would be hard to test got changed at the design stage rather than argued about later.
-
Storefront design
Catalogue, product and checkout designed for a considered purchase on a phone, with the states — empty, error, out of stock — designed rather than improvised.
-
Commerce build
Catalogue, basket and checkout built with the state handled carefully enough to survive a customer opening three tabs and changing their mind.
-
Payments
Payment integration with the failure paths treated as first-class: declines, timeouts, duplicate submissions and refunds.
-
Test coverage
Automated coverage of the commerce logic plus structured manual testing of the paths a customer actually takes, run continuously rather than at the end.
-
Device and performance testing
Real devices and throttled connections, because a storefront that is fast on a developer's machine tells you nothing.
-
Release confidence
A repeatable release process where what is being shipped has already been exercised, so releases stop being events.
Key capabilities
What it does day to day
Six capabilities across the storefront and its assurance.
| Capability | Runs | Refresh | What it does |
|---|---|---|---|
| Catalogue & product | Customer | On publish | Product pages written for a considered purchase, with designed empty and error states |
| Basket & checkout | Customer | Real time | State that survives multiple tabs, delays and changes of mind |
| Payments | Automatic | Per transaction | Declines, timeouts, duplicates and refunds all handled explicitly |
| Automated coverage | CI | Per change | Commerce logic covered automatically, grown alongside the code |
| Device & performance testing | QA | Per release | Real devices on throttled connections, not a developer laptop |
| Release checks | CI | Per release | What ships has already been exercised, so releases stop being events |
Integrations
How the moving parts plug in
Design, build and test ran as one loop rather than three phases.
Design
- Flows and statesIncluding empty and error
- TestabilityConsidered at design time
- Mobile-first layoutWhere the customers are
Build & test together
- Working slicesTestable as soon as they exist
- Automated coverageGrown with the code
- Device testingThroughout, not at the end
Release
- Payment hardening
- Performance verified
- Release checks
- Launch
Because testing had a voice during design, decisions that would have been hard to verify were changed while changing them was free.
Security & data
What protects customers and their payments
A storefront takes card payments and holds order history, so the security work is concentrated where money moves.
-
Payment handling
Card details go to the payment provider. The platform holds a payment reference and an order, never a card number.
-
Server-side pricing
Prices, stock and totals are confirmed server-side at checkout, so what is charged matches what was shown.
-
Duplicate protection
Duplicate submission is handled explicitly, so an impatient customer is not charged twice.
-
Order data
Order history and customer details are scoped to the account they belong to, with no exposure through predictable links.
The brief
Commerce bugs cost money the moment they reach production
A defect in most software wastes someone's time. A defect in a checkout takes money, or fails to take it, and the customer does not come back to try again. That changes how much testing is worth and when it has to happen.
Thursby wanted design, build and testing handled together rather than testing bolted on at the end by people who had not been part of the decisions. That is a harder engagement to run and a much better one to ship.
- Design, development and testing in one engagement
- Checkout and payment paths tested against the awkward cases
- A storefront that performs on real connections
- Defects found before release, not by customers
What eCommerce testing has to cover
- 01Payment success, failure, timeout and duplicate submission
- 02Stock, pricing and basket state under concurrent use
- 03Real devices and real connection speeds, not a fast laptop
- 04Every path a customer can take, including the wrong ones
Process
Testing ran alongside the build, not after it
The only structural difference from a conventional project, and the one that mattered.
-
Design with testability in mind
Flows and states designed so they could be verified automatically, which quietly removes a category of later argument.
-
Build in slices
Catalogue, basket and checkout built as working slices, each one testable as soon as it existed.
-
Continuous testing
Automated coverage grown with the code and manual testing on real devices throughout, rather than a test phase at the end.
-
Payment hardening
Concentrated work on the payment paths, including every failure mode, because that is where commerce defects are expensive.
-
Release
A release that was uneventful, which on an eCommerce build is the whole objective.
Technology
A commerce stack with testing built in
Conventional web commerce technology, with the test tooling treated as part of the platform rather than an add-on.
Storefront
- Responsive storefront
- Catalogue and product pages
- Design system
Commerce
- Basket and checkout
- Order management
- Stock and pricing
Payments
- Payment integration
- Failure handling
- Refunds
Quality
- Automated tests
- Device testing
- Performance testing
- Release checks
Business impact
What the approach delivered
Three consequences of refusing to leave testing until the end.
-
An uneventful launch
Defects were found by the team rather than by customers, which on a commerce build is the result worth reporting.
-
Payment paths that hold
The failure modes that cost real money were exercised before release rather than discovered after it.
-
Fast where it matters
Performance verified on real phones on real connections, which is where the customers actually are.
The result
A storefront that shipped without the usual launch week
Design, build and testing ran as one engagement for a year, so defects were found by the team rather than by customers, and the payment paths were exercised against their failure modes before release.
The client rated the work 5.0 in a verified Clutch review. On a commerce build, an uneventful launch is the result worth reporting.
- Design, development and testing in a single engagement
- Payment failure paths tested, not assumed
- Real devices and throttled connections in the test matrix
- 5.0 rating in a verified Clutch review
What we would insist on for commerce
- 01Put testing in the same engagement as the build
- 02Design the error and empty states, do not improvise them
- 03Test declines and timeouts as carefully as successful payments
- 04Measure performance on a real phone on a real connection
Client review
What the client said about the work
Published on Clutch, where the client is verified and we cannot edit or remove what they write.
-
Web Development
E-Commerce Development, Design & Testing
They're a highly accommodating and skilled team.
Steve Dunlap President, Thursby Software Systems LLC Feb 2022 – Jan 2023 Verified review on Clutch (Steve Dunlap, opens in a new tab)
Free discovery call
Have an idea? Let's turn it into AI-powered software.
Book a free discovery call with our experts. Share your idea and we will help you shape the scope, timeline and budget, under NDA.
- Free consultation
- NDA before we talk
- Transparent estimate