Testing Before Launch, When You Are Not a Tester

Your testing is different from theirs

Developers test that the system does what the specification says. That is necessary and it is not sufficient, because the specification was written by people who did not know everything about your business.

Your job in acceptance testing is to represent reality: the customer with two addresses, the order placed at 4:59pm on a Friday, the member of staff who tabs through a form in a different order.

Write the test list before you see it

If you write tests after using the software, you will unconsciously test what it does well. Write them from your process, in advance, and you test what your business needs.

1. List the five workflows your business runs most often, as numbered steps
2. For each, note the expected outcome in plain terms
3. Add the three exceptions you handle most often
4. Add the two things that would be worst if they went wrong

That list is your acceptance test, and it should be agreed with the supplier before the build finishes rather than sprung at the end.

Use real data, not tidy data

Test data created for testing is always well-behaved. Real data has apostrophes in surnames, addresses with four lines, phone numbers in five formats, product names with symbols, and records created in 2011 by someone who has left.

Use a copy of your real data for acceptance testing, anonymised if needed. Nearly every serious defect we have seen discovered late was found by real data doing something test data never did.

Test the failures deliberately

• Submit a form with required fields empty — is the message helpful?
• Enter obviously wrong data — a date in the past, a negative quantity
• Interrupt something halfway — close the tab mid-process and go back
• Try to do something you should not be allowed to do, with a low-privilege account
• Use the browser back button, which breaks more applications than anything else
• Do the same thing twice quickly — double-clicking submit is not exotic

Test on the actual devices

Test on the machines your staff use, not the newest laptop in the office. If the warehouse uses a five-year-old tablet on patchy wifi, that is the test environment that matters.

Test on a phone as well, even for internal systems. Someone will use it on a phone regardless of what the specification assumed.

Record defects usefully

“It's broken” wastes a round trip. Record what you did, what you expected, what happened, and a screenshot. Note the account you were using and the time, because logs are searchable by both.

Agree severity levels with the supplier — blocking, major, minor, cosmetic — and which ones must be fixed before launch. That conversation before testing prevents an argument during it.

Frequently asked questions