Xyntherium Verify · 19 October 2026

verify_answer: check what your assistant drafted against the document it was given

Support assistants usually get it right. When they get it wrong, it tends to be quiet: the answer is fluent and nearly complete, and it leaves out the one condition that mattered. verify_answer checks a drafted answer against the passages it was based on, before anyone sends it.

The same answer, before and after the check

The first line also names “Unworn items” and the tags condition. That is noise: the checker split the answer into two statements, and the second (“returnable within 30 days of delivery”), read on its own, lacks them, though the answer as a whole states both. The final-sale exception is the real miss.

Returns and refunds

Unworn items can be returned for a full refund within 30 days of delivery. Items must have their original tags attached.

Shipping costs are not refunded. Return shipping is free for Northwind members; other customers pay a $7 return label fee, deducted from the refund.

Final-sale items, including anything bought in the clearance section, cannot be returned or exchanged.

Refunds are issued to the original payment method within 5 business days of receiving the return. Gift purchases are refunded as store credit.

How often this happens depends on the model and the setup. We asked claude-sonnet-5-5 the same question three times, and it kept every condition each time. claude-haiku-5-5, limited to 20 words, dropped the final-sale exception in one of five runs; that is the answer above, unedited. Production assistants run on cheaper models, under length limits, on retrieved chunks that sometimes miss the paragraph with the exception. A check that runs on every answer catches the one in five.

Reproduce it

The check, without an API key (this exact input is a published demo):

npx -y @modelcontextprotocol/inspector --cli https://www.xyntherium.com/mcp --transport http \
  --header "Authorization: Bearer $XYN_API_KEY" \
  --method tools/call --tool-name verify_answer \
  --tool-arg 'question=can I get a refund on the jacket I got 2 weeks ago? it doesn'\''t fit' \
  --tool-arg 'answer=Yes, if it'\''s unworn with tags attached, it'\''s returnable within 30 days of delivery; non-members pay a $7 fee.' \
  --tool-arg 'sources=[{"id":"returns","title":"Northwind Outfitters: Returns and refunds","text":"Returns and refunds\n\nUnworn items can be returned for a full refund within 30 days of delivery. Items must have their original tags attached.\n\nShipping costs are not refunded. Return shipping is free for Northwind members; other customers pay a $7 return label fee, deducted from the refund.\n\nFinal-sale items, including anything bought in the clearance section, cannot be returned or exchanged.\n\nRefunds are issued to the original payment method within 5 business days of receiving the return. Gift purchases are refunded as store credit."}]'

The draft (results vary from run to run; that is the point):

claude -p --model claude-haiku-5-5 "You are the support chat assistant for Northwind Outfitters. \
Answer the customer in one short sentence (under 20 words), using the policy below.

<the policy>

Customer: can I get a refund on the jacket I got 2 weeks ago? it doesn't fit"

How it decides

The answer is split into statements. Each one has to be found in the passages, by code, with its conditions intact: deadlines, exceptions, limits, eligibility rules, numbers and dates. A model may propose which passage supports a statement, but its quote counts only if the code finds it in the source. On a fresh test of 300 answers, AI judges let through about 1 in 4 of the flawed ones and Verify’s rules let through 1 in 28 (the full comparison, and why the test favours Verify).

Three ways to run it, one free tier

• MCP: https://www.xyntherium.com/mcp, for Claude Code, Claude Desktop, Cursor and VS Code; setup. For Claude, a plugin adds a skill that knows when to call it: /plugin marketplace add augustbrookz/verify-gate.
• API: POST /v1/verify; docs.
• Verify Gate: a GitHub Action that re-checks saved answers when a document changes in a pull request; github.com/augustbrookz/verify-gate.

Free: 1,000 checks a month across all three, no card. Pricing.

What it does not do

It checks against the passages you pass, nothing else: if the right paragraph is missing from them, it cannot tell. When the passages do not settle a statement, it says so (ASK A PERSON) instead of guessing; on the benchmark it sends about a third of answers to a person. Verdicts describe whether the supplied passages support each claim. They do not establish whether any claim is true in the world.