A trial with nothing to test on
A firm agrees to a four-week trial. The first day goes well: training, logins, enthusiasm. Then the fee earners ask what they should test it on. Client documents are off limits until the product has passed the firm's security review, which has not started. Someone finds a couple of sample contracts online. A trainee runs them through, the output looks fine, and nobody tries anything else.
At the end of the trial, the feedback is polite and vague: looks promising, hard to judge without real work. The firm's decision is postponed until the security review is done, which could be months.
Why trials run out of material
Law firms protect client information carefully, and loading it into an unapproved product would breach their own policies. That is right. It leaves trials with little to work on.
- Real client documents are off limits until security review and contracts are complete.
- Public sample documents are few, generic and too clean.
- Fee earners have no structured tasks, so they try one thing and stop.
- The firm's own precedents, which are not client confidential, are not loaded because nobody thought of it.
- Trial and production are the same environment, so firms are nervous about anything they upload.
What thin trials cost
A trial that produces no real evidence delays the decision, and delayed decisions often become no decision. Fee earners who saw the product on unrealistic documents underestimate what it can do, or overestimate it and are disappointed later. Your team spends onboarding and support effort on a trial that teaches neither side much.
It also skews what you learn about your own product. Clean public samples hide the drafting quirks that cause real errors, so the trial reports no problems, and the first real difficulties appear after the firm has signed, on live matters, where they matter much more. A harder trial is better for both sides.
How we build trials firms can learn from
What we build gives firms realistic material to test on, without any client data.
- A separate trial sandbox, isolated from production, so firms know nothing uploaded there touches live systems or other firms.
- Synthetic matters per practice area: realistic document sets such as a lease with schedules and side letters, a share purchase with disclosure letter, or a disclosure set for a dispute, generated and then checked by a lawyer for realism.
- Deliberate difficulty: unusual drafting, buried clauses, poor scans and inconsistent definitions, because that is what real matters contain.
- The firm's own precedents and standard documents loaded where the firm agrees, since these are often not client confidential and show the product working with the firm's own drafting.
- Structured test tasks for each practice group, mirroring real work, with a simple way for fee earners to record how the product did.
- A trial report summarising tasks attempted, feedback and issues, and a clean path from sandbox to full deployment once review and contract are complete.
| Trial material | Source | Confidential client data? |
|---|---|---|
| Synthetic lease set | Generated and lawyer-checked | No |
| Synthetic share purchase documents | Generated and lawyer-checked | No |
| Firm's own precedents | The firm, if it agrees | Usually no, the firm decides |
| Public documents | Published sources | No |
| Real client matters | Only after approval | Yes, not used in trial |
What the firm may load into a trial is its own decision under its policies. We make sure there is plenty worth testing without client data.
A trial with real work to do
The next firm's trial starts with a sandbox containing synthetic matters for their real estate and corporate teams, and their own standard lease loaded with the knowledge team's agreement. Each fee earner has three test tasks that match their usual work. A lease with a break clause buried in a side letter catches out one fee earner's own first read, and the product finds it. By the end of the trial, the firm has recorded results on each task, and the decision meeting has something concrete to discuss.
Are your trials running on thin air?
- Fee earners in trials test your product on one or two public samples.
- Trial feedback is vague and positive, and decisions are postponed.
- Your trial runs in the same environment as production.
- Firms' own precedents are not used in trials.
- Fee earners have no structured tasks during the trial.