Keep exceptions visible
Use the cases and scoring guidance to compare performance consistently and record failure patterns.
AUT-017 · Automation
We create 40 representative cases with scoring guidance and failure examples for one task.
Use the cases and scoring guidance to compare performance consistently and record failure patterns.
We create 40 representative cases with scoring guidance and failure examples for one task.
In the AI output evaluation set, follow “Evaluation case”, “Expected quality” and “Scoring rationale” to check the handover against your original question.
A few successful examples say little about how an AI system behaves with ambiguous or unusual inputs. The service is intended for operations and systems owners. Use the cases and scoring guidance to compare performance consistently and record failure patterns. It is useful when your team can identify the decision the work needs to support and name someone to review what comes back.
Approved examples, permitted source material and the named human reviewer. Before work starts, explain how the result will be used, which inputs are already approved and whether access or material must come from someone else. Flag gaps in the current information early so they can be handled within the scope or recorded for follow-up.
Choose representative and difficult cases, Write forty evaluated examples and Document scores and failure types. The work takes place remotely through an agreed channel. We identify questions that affect the outcome, confirm where your team needs to make a decision and explain any assumptions that remain at handover.
Indicative delivery after agreed inputs are complete: 3–7 working days. Platform, AI usage and connector subscriptions are separate. Timing may also depend on access, client review or decisions outside the agreed scope. Confirm who owns those dependencies before using the estimate for a launch or other commitment.
Begin by checking the ai output evaluation set against the original question and the information your team supplied. Decide whether you can act on the result directly or need to collect evidence, obtain approval or commission a separate implementation. That distinction helps you turn the assignment into a practical next step.
Work is limited to approved inputs and authorised systems. Test normal and exceptional cases before any rollout. AI outputs need human review and should not be treated as final decisions.
Describe the extra work when you enquire. Any change to the published scope, file limits or timing needs separate written agreement.
Wait for instructions about the agreed channel. Share only the material needed for the assignment and confirm that your organisation has the right to share it.
We configure a drafting workflow for one document type and test it on 20 approved examples.
We build a read-only pilot over 25 approved documents with source citations and test questions.
Tell us about your business and the outcome you want to achieve.