A customer enquiry is awaiting a reply. The AI assistant drafts a goodwill commitment, even though the applicable rule for this case is not available. The wording sounds definitive. For the company, this may create an obligation that nobody intended to enter into.
This is an illustrative situation, not a client case. But it illustrates a decision that must be resolved before an assistant is used: may the system replace missing business rules with a plausible answer?
My answer is no. If a necessary basis is missing, the assistant must identify that gap, ask a targeted question or escalate the case. For the time being, an unanswered question may be the right result. What matters is whether the work then continues in a controlled way.
A high response rate is not evidence of quality
A paper published in Nature in 2026 makes the trade-off visible. In a defined SimpleQA test involving 4,326 short factual questions, the o4-mini model gave incorrect answers to 76.8% of all test questions. Only 2.6% were left unanswered. These are results from this specific historical model test, using fixed settings. They are not an error rate for today’s AI assistants in organisations, nor a comparison of their customer-service quality. 1
The paper’s more important point concerns evaluation: if only correct answers are rewarded and non-answers are treated in the same way as incorrect answers, this can incentivise guessing. For a business, that distinction matters. A request for clarification costs time. A fabricated interpretation of a contract can have a very different consequence.
I would therefore not make the highest possible response rate the sole objective. The responsible business lead must define how much uncertainty is acceptable for which task. An internal draft of an idea requires different limits from a commitment to a customer.
Check the basis, not the wording
In the goodwill example, it is not enough for the assistant to have found a document. It must apply to the relevant customer group, be current and actually cover the exception in question. General guidance to be customer-focused is not authority to promise a specific service.
If only one detail of the order is missing, a targeted question can clarify the case. If two apparently valid versions of a rule conflict, a business decision is needed. If a source cannot be accessed, the system must not infer either permission or prohibition from that fact. Nor may it bypass access restrictions in order to provide an answer anyway.
The output should make clear which basis is missing and what happens next. A general warning beneath a commitment that still sounds definitive achieves little. The assistant can summarise information that is already evidenced. It must leave the unresolved decision open.
The voluntary US reference framework NIST AI 600-1, published in 2024, supports this approach. It recommends assessing sources and citations before use and during ongoing monitoring, as well as verifying data provenance. Systems designed for safety should also be able to fail safely beyond the limits of their knowledge. This is guidance for design and evaluation, not a certification or a legal guarantee. 2
Test cases that must not be answered
First, I would select a tightly bounded process. For that process, the person with business accountability defines the approved sources, required information and expected behaviour. The assistant is then tested in an isolated test environment with cases in which a direct answer would explicitly be wrong:
- ◆The relevant rule is missing. The system identifies the gap instead of inventing a rule.
- ◆Two sources conflict. The case is passed to the named responsible business function.
- ◆Only an expired rule is available. It is not used as the current basis.
- ◆A decisive case detail is missing. The assistant asks for precisely that information.
The test must also include cases that can properly be answered. Otherwise, a system that refuses every question and is of little help in operation could pass. The assessment therefore covers incorrect commitments, justified requests for clarification and unnecessary refusals. How serious these errors are is determined by the leadership team on the basis of the consequences in the relevant process, not by a general benchmark rate.
A handover requires more than pointing to a human. The responsible role must be defined, must actually receive the case and must be able to see the unresolved question with its context. There must be an agreed fallback route if the case is not dealt with. Where a commitment has material consequences, execution remains blocked until the issue is clarified. These arrangements are reviewed again after changes to rules, sources or the model.
This is one small, concrete component of a controllable AI Operating System. If decision rules of this kind first need to be defined and prioritised across several processes, the Executive AI Transformation Blueprint can create the management architecture required. The three-week Blueprint plans accountabilities, controls and implementation. It does not replace subsequent testing in live operations.
At the next review, I would therefore not ask only how many enquiries the assistant has answered. I would ask to be shown a case that it has left unresolved for good reason, then check whether that case has reached the right function.