Vendor demos show a tool at its best. Before you pay for one, run your own small test. This is a shorter version of the method behind our ratings.
1. Pick one real job
Choose a task you do every week, such as reconciling a bank export, tailoring a CV or writing up meeting notes. A narrow task gives a clear answer; "is it good at writing?" does not.
2. Build fake but realistic data
Never paste client or customer data into a tool you are trialling. Make a small fictional version instead: ten bank lines, a one-page resume, a short meeting transcript.
3. Plant traps
The most useful part of the test. Hide two or three things a careful person would catch:
- a duplicate transaction or a refund with the wrong sign;
- a job ad skill the candidate does not have, to see if the tool invents it;
- a decision in a meeting that was discussed but not agreed;
- a blank cell, to see if the tool makes up a value.
Write down the correct answer before you start.
4. Write a simple scorecard
Split 100 points across what matters to you. Ours usually look like this: accuracy 30–35, honesty (no invented facts) 20–30, clarity 15–20, safety 15, friction 10. Friction covers sign-up walls, usage limits and upsells.
5. Run the same prompt in each tool
Paste the identical prompt and data into each tool. Save the full output or a screenshot. Note the plan and model, because free and paid tiers can behave differently.
6. Score, then run it again
Mark each output against your answer key. AI tools vary between runs, so repeat the test at least once before deciding. A gap of two or three points is noise.
What to watch for
- Confident invention: the most costly failure. A tool that says "I can't tell from this data" beats one that guesses.
- Unasked-for advice: tax or legal commentary you did not request.
- Locked exports: work you cannot download on the free plan.