How We Test Software Before You See It

Why we test before demos, not after

A demo where something breaks costs disproportionate confidence. The bug might be trivial; the impression is that the system is fragile, and that impression takes three good demos to repair.

So the last thing before every demo is a run through the exact path we will show, on the environment we will show it on. It takes twenty minutes and it has saved a great many awkward calls.

What gets automated

• Business logic. Pricing rules, validation, calculations, state transitions — anything with rules worth getting right.
• Integrations. Against recorded responses, including the error responses.
• Critical journeys. A handful of end-to-end tests covering the paths that must never break.
• Regressions. Every bug we fix gets a test, so it cannot come back quietly.

We do not chase a coverage percentage. We cover the code where being wrong costs money, and we are relaxed about the code where it does not.

What stays manual

Anything about judgement or feel: does this journey make sense, is the error message helpful, does it work on a real phone in real light, is the wording right.

• Exploratory testing — using it like a person trying to break it
• Real devices, including an older mid-range Android
• Accessibility with a keyboard and a screen reader
• The complete journey a new user takes, from nothing

Testing the unhappy paths harder

The happy path gets exercised constantly during development. The failure paths get exercised in production, by a customer, at the worst moment — unless somebody tests them deliberately.

1. What happens when the third-party API is down?
2. What happens on a slow or dropping connection?
3. What happens when two people edit the same record?
4. What happens with an empty state, and with ten thousand records?
5. What happens when a required field arrives null that never has before?

We write these into the test plan as first-class cases. The most common production incidents in software we have inherited were all in this list.

The environments

Staging matters more than teams expect. A client testing on the same environment developers are pushing to sees half-finished work and forms an unfair impression of stability.

What we ask you to test

Your team should test the things only you know: is this how the process actually works, are these the right terms, would a customer understand this, is this edge case real.

We will give you a short checklist per sprint rather than asking you to test everything. A focused list of eight things gets done properly; an open invitation to test the system gets a five-minute click-around.

Frequently asked questions