For compliance
Valid structure is not evidence of a valid decision.
A response can pass format checks and still produce the wrong regulated or policy outcome. Schema validation controls syntax; it does not validate reasoning.
Technical note / AI governance / July 2026
Valid JSON can still be confidently wrong.
We tested five ways of asking the same model to apply the same policy. Better prompts and richer schemas improved the result, but none made generated reasoning dependable. The only design with zero wrong decisions separated the work: the model extracted facts, deterministic code applied policy, and the model narrated the result.
Customer messages25
Runs per message200
Output designs5
Measured generations50,000
Fully correct decisionHeld-constant policy
Validators-first prompt + schema81.8%
Best schema-only design89.7%
Extract → code → narrate100.0%
01 / Why this matters
The dangerous output was not broken. It was persuasive.
A malformed response is easy to detect. The harder governance problem is a fluent answer that follows the schema, agrees with itself and applies the policy incorrectly.
For governance
Generated predicates are not automatically an audit trail.
Fields labelled as checks or reasons may reflect the same underlying holistic guess. They should not be treated as independently verified control evidence.
For product
More model reasoning can cost more and perform worse.
The most reliable design used fewer tokens than every scratchpad approach. The architecture improved quality and efficiency together.
For engineering
Constrained decoding solves a different problem.
Grammar enforcement maintained valid, ordered output as schemas became complex. It did not change which decision the model reached.
For risk
Failure direction matters as much as failure rate.
Schema-only failures confidently approved ineligible cases. The deterministic pipeline converted extraction uncertainty into a route for human review.
For leadership
The control is architectural, not rhetorical.
The result was not achieved by another prompt instruction. It came from assigning different work to the model and to deterministic software.
02 / Headline result
Each design change closed part of the gap. Code closed the rest.
A validators-first schema reached 82.6% decision accuracy. Adding explicit working improved some cases but plateaued below 90%. Moving the policy decision into deterministic code produced 10,000 correct decisions from 10,000 runs in the bounded evaluation.
The model was best used at the edges of the decision.
Throughout the study, the model extracted information from the data set near-perfectly. Narration remained consistent with the chosen outcome. The unreliable middle was policy computation: sums, threshold bands, boundary conditions and combinations of predicates.
Core finding
Schemas should capture facts. Code should apply policy.
The model reads the customer’s message and explains the result. Deterministic software performs the decision that must be repeatable and auditable.
03 / Finding one
The baseline failed silently, confidently and repeatedly.
The error was concentrated in cases requiring multiple prices to be added before comparison with a limit. Directly stated over-limit totals were handled almost perfectly; multi-part totals were not computed before the first decision field was emitted.
No malformed JSON. No visible hesitation.
On four scenarios, an ineligible customer was booked 400 times out of 400. The model returned the expected fields, used the expected reason code and wrote a response fully consistent with its own wrong decision.
400 / 400
Confidently incorrect bookings across the concentrated failure scenarios.
Schema-valid outputPolicy-invalid decision
{
"criteria_met": true,
"reason_code": "eligible",
"action": "book_appointment",
"response_text": "I can help arrange that for you."
}
// JSON valid ✓
// Required fields ✓
// Internally aligned ✓
// Policy decision ✕Governance consequence
A system that checks only parsing, required keys and agreement between response text and action would approve this output. The control has to validate the decision itself—not merely its presentation.
04 / Finding two
A scratchpad fixes only what it forces onto the page.
Adding a calculation field before the decision repaired the obvious arithmetic failures. Clear multi-part over-limit cases moved from 0% to 99.5% correct because the model was now required to write down the sum.
But classification into the intermediate policy band remained unstable. The model could calculate £6,500 + £4,250 = £10,750 correctly and still assign the harshest category instead of the reduced-scope alternative.
Clear multi-part cases0 → 99.5%
Written arithmetic repaired the failure it directly exposed.
£10,750 boundary case≈0%
The sum was right while the policy band was wrong.
Token cost128
Up from 81 tokens for the baseline design.
Showing the arithmetic did not turn the schema into a dependable decision process. It moved the error to the next unresolved step.
05 / Finding three
A schema is not a computation graph.
The fourth design represented every policy predicate as a required field: extracted age, extracted total and one boolean for each threshold comparison. It produced the best schema-only result—and the clearest evidence that generated fields should not be treated as independently computed checks.
What the fields appeared to show
A sequence of verified predicates
Age extracted → total calculated → £10,000 comparison → £12,000 comparison → action selected.
What the outputs demonstrated
Correlated samples from one holistic judgment
Fields copied one another, absorbed the overall verdict and sometimes disagreed with the action they supposedly determined.
Copied comparisons99.5%
The £12,000 comparison matched the £10,000 comparison, even when their true answers differed.
Wrong-copy cases97.3%
Runs where the two comparisons should differ but were emitted identically.
Action/field agreementas low as 8%
The final action did not reliably follow the model’s own preceding booleans.
The fields looked like an audit trail. They were not one.
06 / Finding four
Enforced decoding buys validity, not correctness.
Decision accuracy was effectively unchanged with and without grammar enforcement in every design. Its value emerged as schemas became more complex: it guaranteed parseable structure and field order when unenforced generation began to degrade.
What enforcement guarantees
A dependable interface.
- Valid JSON matching the schema
- Required fields and types
- Deterministic field ordering
- Fewer retries and fallback branches
100.0%schema validity under enforcement
What it does not guarantee
A correct policy outcome.
- Correct arithmetic
- Correct threshold classification
- Independently computed predicates
- Agreement between fields and causal logic
±0.1ptdecision difference with vs without enforcement
On the more complex unenforced schemas, validity fell to 97.9%—approximately one malformed response in fifty. The measured throughput cost of enforcement was approximately 10–20%. It remains necessary infrastructure; it simply solves a different problem.
07 / Finding five
Extract facts. Apply policy in code. Narrate the result.
The final design removed the decision from the model - this is the format process used within our own agent pipelines. The same model extracted bounded facts, approximately twenty lines of deterministic code applied the policy, and the model wrote the customer-facing response from that result.
The architecture changed both the rate and the failure mode.
Every scenario that had failed catastrophically—multi-part sums, band boundaries and the exact age boundary—reached 200 correct decisions from 200 runs.
When extraction was uncertain during development, the code could route the thread to human review. Schema-only designs instead produced a fluent, confident answer from a wrong decision.
Decisions correct10,000/10,000
Errors0
Tokens/eval97
Residual risk becomes legible
A recurring extraction slip read “£9,950” as “£9,990” on one prompt. The fix is a source-grounding check: verify that every extracted number appears in the original message. The remaining risk is local, observable and testable.
01 / ExtractLLM
Read bounded facts
Customer age, stated prices, requested date and requested time. The model is forbidden from deciding or summing.
02 / DecideCode
Apply the policy deterministically
Sum values, test both thresholds, check age bounds, normalise dates and select the defined action.
03 / NarrateLLM
Explain the decided outcome
Generate a natural response from the code-owned action and reason, without reopening the decision.
08 / Governance implications
What this changes in a production control framework.
The experiment does not argue against language models or structured outputs. It defines where each belongs in a dependable system.
Control 01
Do not equate schema validity with decision validity.
Parsing and field checks are interface controls. Test policy decisions separately against known-correct outcomes.
Control 02
Do not present generated predicates as verified reasoning.
If a field matters to the decision, compute or verify it independently before treating it as audit evidence.
Control 03
Assign deterministic policy to deterministic software.
Use code for arithmetic, thresholds, eligibility, dates, idempotency and other rules that must be repeatable.
Control 04
Use the model where its capability is strongest.
Language models are effective at extracting facts and producing natural responses when the decision boundary is controlled elsewhere.
Control 05
Make uncertainty fail safe.
Extraction checks and explicit escalation should stop incomplete facts from becoming confident customer-facing actions.
Control 06
Evaluate the composed system.
Model metrics alone are insufficient. Test extraction, code, narration and handoff behaviour together across boundaries and regressions.
09 / Results table
Five output designs, measured on the same task.
The customer messages, policy and model were held constant. The constrained condition provides the comparable decision scores below; the unenforced condition isolates schema validity.
| Output design | Decision correct | Fully correct | Schema-valid unenforced | Response consistent | Tokens / eval |
|---|---|---|---|---|---|
| 1. Prompt + validators-first schema | 82.6% | 81.8% | 100.0% | 99.9% | 81 |
| 2. + Calculation scratchpad | 88.1% | 87.6% | 98.5% | 99.2% | 128 |
| 3. + Decomposed limit gate/tier | 85.7% | 85.5% | 97.9% | 99.8% | 135 |
| 4. + Full predicate table | 89.7% | 89.7% | 97.9% | 99.6% | 166 |
| 5. Extract → code decides → narrate | 100.0% | 100.0% | 100.0% | 99.8% | 97 |
“Fully correct” requires the action, reason code, criteria flag and appointment outcome all to be correct. Enforced decoding held validity at effectively 100% across all designs; the validity column reports the corresponding unenforced condition. Rounded values are shown.
10 / Method and boundaries
Designed to isolate the output architecture.
One model, one server, fixed prompts and a known-correct policy. Only the structure of the requested output and the use of enforced decoding changed between conditions.
ModelLlama 3.1 8B
Instruct model served through one vLLM environment.
Scenarios25
Customer messages with independently verified expected outcomes.
Sampling200×
Generations per message, per condition, at temperature 0.7.
Total50,000
Five designs, each tested with and without enforced decoding.
What the study supports
- Structured fields did not behave as a dependable chain of computations on this task.
- Grammar enforcement controlled validity and order, not decision correctness.
- The composed extract/code/narrate design eliminated observed decision errors in this bounded test.
- Architecture produced a safer and more legible failure surface.
What it does not prove
- That every policy task will achieve 100% accuracy with the same architecture.
- That extraction is error-free or can be used without grounding checks.
- That 10,000 successful runs establish universal or permanent reliability.
- That deterministic code removes the need for monitoring, regression testing or human escalation.
Differences were tested using Fisher’s exact test by scenario group. The full technical write-up contains scenario-level results, hypotheses, statistical comparisons and implementation detail.
Want to run the evaluation yourself? Contact our team and we'll fork the repo.
Schemas capture facts.
Code applies policy.
For decisions that must be repeatable, explainable and safe at the boundary, the right question is not how to make the model reason harder. It is which parts of the work should never have been probabilistic.