Confidently Wrong: Testing a World Cup Rules Chatbot
Building an AI assistant on the official FIFA 2026 rulebook, then measuring where it still gets things wrong
The version that answered the most questions correctly was also the one that invented an answer to every single question it should have refused.
The 2026 World Cup runs on a new 48-team format, and the rules for separating teams level on points are a sequence of steps that have to be applied in the right order. Ask a general chatbot about either and it answers from whatever tournament it saw most often while being trained. I tested one: it got nine of twelve rules questions wrong, including stating the yellow card rule as the exact opposite of what the regulations say, and it wrote every wrong answer in the same confident tone as the right ones.
The usual fix is to hand the model the actual rulebook and require it to answer only from what it finds in there. That worked. Correct answers tripled, from a quarter to three quarters, and about three quarters of the confident mistakes disappeared. But it created a new problem. When I asked questions the rulebook genuinely cannot answer, like who will win the tournament, this version answered every one of them anyway instead of admitting it could not.
Adding instructions and two worked examples fixed most of the refusals, and scored slightly lower on raw accuracy as a result. That trade is the finding I would defend hardest. Two reasonable ways of scoring the same system pointed at two different versions to ship, and the accuracy score on its own pointed at the one that makes things up. The same trap showed up in miniature on the tie-break question, where my measure of whether the assistant had found the right page of the rulebook read a perfect 100% for a question it still got wrong.
Why it matters
Anyone buying or building an AI assistant gets shown an accuracy score. This is a worked example of why that number on its own can point you at the wrong system. The highest-scoring version here was the one most willing to make things up when it should have said it did not know. And a fluent wrong answer reads exactly like a correct one, so nobody using it could tell the difference without being shown the source.
Two ways of scoring the same assistant. The middle version answers the most questions correctly and never once admits when it cannot answer.
Confident errors by cause. Supplying the rulebook cuts them from 8 to 2, and flips every remaining one from invention to cases where the right page was retrieved and the answer was still wrong.
Dataset, tools and how it was done+
Dataset: Official FIFA World Cup 2026 regulations (98 pages), with a purpose-built 15-case evaluation set
Tools: Python · Ollama · sentence-transformers · PEFT / LoRA
- Source was the official 98-page FIFA World Cup 2026 regulations, with a purpose-built 15-case evaluation set recording the answer page and key facts for each question, plus three questions where refusing is the only correct response
- Compared three configurations on the same questions: no rulebook, rulebook only, and rulebook plus an engineered prompt with worked examples
- Correctness tripled from 0.25 to 0.75 once the rulebook was supplied, and confident errors fell from 0.727 to 0.182
- The rulebook-only version refused 0 of 3 unanswerable questions; adding prompting raised that to 2 of 3 while costing a little accuracy, because refusal is a prompting property rather than a retrieval one
- Swept passage size, overlap and number of passages retrieved; 300 words with 20% overlap was the only setting that found every answer
- Measured that the embedding model silently truncates any passage over 256 tokens, which affects 100% of passages at the chosen setting, so the sweep found the best performer despite that handicap rather than one that avoids it
- An ablation isolated the cause of the improvement: the worked examples, not the written instructions, produced correctly ordered multi-step answers, and the same examples caused the only regression
- Adversarial testing: direct prompt injection succeeded against all three versions, a poisoned passage planted in the index was ignored, and asking the assistant to repeat its instructions leaked the retrieved source text
- Extension: LoRA fine-tuning of the retriever left accuracy unchanged, and with only 7 training pairs the test was not sensitive enough to detect an effect in either direction