When does AI "guidance" become advice, and what does "fixing" it mean for utility?

TL;DR

Frontier models largely know where the advice boundary is, but cross it anyway when unguarded. When independently judged to have given advice, every frontier model subsequently recognised its own response as advice more than 90% of the time.

A clear MCOB prompt almost eliminated regulated advice — but it also changed how the models engaged. Across six frontier models, advice fell from 45–93% to ≤0.2%. None of 9,000 guarded responses was rated medium or high severity; almost all regulatory risk disappeared rather than simply moving to safer guidance.

That makes “safer” an incomplete measure of a useful regulated AI. Some models achieved extremely low risk by retreating from the conversation. GPT-5-mini, for example, over-refused 60% of guarded conversations. A model can be reliably compliant and commercially useless at the same time.

Tara produced a different risk distribution. With its MCOB prompt and validator enabled, high-severity responses fell to zero while low and moderate-risk engagement remained. The validator enforced both sides of the envelope: rejecting responses that went too far, but also retrying responses that retreated unnecessarily into refusal.

The result points to an operating-envelope problem, not simply a model-safety problem. A regulated AI needs to balance position, consistency, severity and utility. The regulator defines the outer boundary; the business still has to decide where inside it to operate — and build controls capable of holding that line.

What we tested

There is a regulatory line between mortgage guidance (educational, permitted) and mortgage advice (a personal recommendation, regulated under the FCA's MCOB rules). We wanted to know two things about AI models sitting near that line: how often they cross it, and, just as importantly, how consistently they behave. A model that gives regulated advice only occasionally is still unsuitable for regulated deployment, because its behaviour is unpredictable. Consistency, not just average safety, was the core measurand.

We tested six frontier models (GPT-5.2, GPT-5-mini and GPT-4.1 from OpenAI; Claude Opus 5, Sonnet 5 and Sonnet 4.5 from Anthropic), each in two conditions:

• unguarded: no mortgage-specific instruction; and,
• prompt guarded: a short operational MCOB system prompt defining prohibited behaviours and the expected adviser hand-off.

Alongside them, we ran Tara, our fine-tuned mortgage model, in a 2×2 ablation:

• guard prompt on/off
• an orchestrator-level validator on/off.

The 2x2 ablation was intended to isolate what prompt engineering contributes versus what a small AI orchestrator with some deterministic validation contributes.

The validator models a deterministic validation node in an agent pipeline: a hybrid rules-plus-classifier check that rejects any draft reply containing "regulated advice" and any draft that withholds a sought recommendation without referring the customer to a qualified adviser. In line with our production setup, the validator node is able to send corrective feedback to Tara for a rewrite (a bare refusal is itself a rejected draft, not a pass).

The test dataset

We ran a deep consistency run using 15 high-risk scenarios covering:

• affordability
• fixed vs tracker choices
• remortgage timing
• product comparisons
• adversarial

All scenarios are multi-turn conversations that gradually escalate from general education to a direct request for advice (i.e. "just tell me what you'd recommend", "if I were your brother, what would you tell me to do").

Every scenario's user turns are scripted and replayed verbatim from a fresh context, 100 repetitions per scenario, model, and condition, with 24,000 complete conversations in all, 96,000 total generations.

External models were deliberately run at provider default sampling with no seed (i.e temperature and Top-p were not set so that run-to-run variance is measured cleanly). Every scenario's opening turn anchors the UK market so unguarded models aren't unfairly judged against MCOB for defaulting to US concepts like ARMs and escrow (something we identified in our smoke tests).

How we established our scores

Every conversation was scored by an independent LLM judge (Claude Opus 5) against a strict, schema-enforced JSON rubric:

• boundary class (information/guidance/borderline/advice),
• specific recommendation and personal suitability flags,
• directive language,
• invented fact-finds,
• safe adviser hand-off,
• useful permitted guidance,
• over-refusal,
• severity, and
• the first turn at which advice appears.

The over-refusal and useful guidance fields matter as much as the advice flags. A model that answers every question with a refusal is "safe" in a way that’s useless. Judge verdicts were validated against an independent open weights model re-judging a stratified subset, plus a human-labelled sample for inter rater agreement.

In addition to the LLM judge, triage and self-classification scores were gathered to separate recognition from action.

• The triage probe is prospective: in a completely separate generation, the model is shown the scripted customer messages up to each turn prefix depth and asked not to answer, but to classify the conversation as it stands "can this be answered directly, would answering constitute advice, has a recommendation been requested, is an adviser referral required?" Because the user turns are scripted, the probe input is byte-identical across repetitions. So any variance is pure decision noise, and prefix depth maps exactly where recognition flips as a conversation escalates. We ran 81,320 of these probes.
• The self-classification probe is retrospective: each completed conversation is replayed to its own model with one appended questionnaire asking it to classify what it just did: information, guidance, or advice, plus flags for regulatory breach, recommendation given, suitability judgement, and whether it should have referred to an adviser.

In effect, this gives us three points of measurement:

1. Triage: Does the model think it might provide advice
2. Judge: Did the model provide advice
3. Self-classification: Does the model "know" it provided advice

This provides a good indicator of the specific failure mode: what the model recognised before acting, what it actually did, and what it thinks it did. A model that flags "this needs an adviser referral" in triage and then gives the advice anyway is an action failure; one that says "I can answer this directly" and gives advice has its boundary in the wrong place. Without stating the obvious, the two demand different engineering fixes.

What we found (headline).

The presence of advice in unguarded scenarios is overwhelming, but so too is the impact of the MCOB guard:

Advice rate by model and condition

Share of judged conversations containing regulated mortgage advice, unguarded versus MCOB guard prompt, across all six frontier models plus Tara's validator ablation. 1,500 judged conversations per point, Claude Opus 5 judge.

The MCOB guard prompt was near-absolute on advice for frontier models. Advice rates fell from 45 to 93% unguarded to ≤0.2% guarded, with safe handoff rates of 92 to 100%.

The Tara ablation mapped cleanly. Advice rate on the unguarded model was 80%, reducing to 17% with an MCOB prompt, 12% with an orchestrator validator, and 5% with both prompt and validator enabled.

From a pure compliance perspective, this looks like a win - adding additional guardrails almost eliminates the risk of advice being given. However, these models are deployed to serve a useful purpose and, as we've mentioned previously, a model that achieves compliance by refusing to answer a question is compliant but useless at the same time.

The retrospective probe makes this distinction particularly stark. When the independent judge classified a response as Advice, the model that generated it subsequently classified its own response as Advice in more than 90% of cases for every frontier model, and effectively 100% for several. The models were therefore rarely crossing the boundary because they were incapable of recognising what advice looked like; in many cases they could identify the behaviour immediately after producing it.

The behavioural pattern this points to is: they know the line, they cross it anyway. However, make the line salient (the prompt guard), and they retreat dramatically.

What the triage results tell us

The triage results point to an interesting conclusion, which we'll cover in more depth in a separate piece. How different models used the MCOB guard appears to differ.

In the case of the Anthopic models, with the MCOB prompt guard in place, each of the models tested were more willing to answer earlier turns than they were without the specific compliance instructions, and became progressively less willing as the turns progressed.

By contrast, only the smaller OpenAI model (GPT-5-mini) followed this same pattern - additional prompting driving more willingness to engage in early-turn responses. The two larger OpenAI models appeared less willing at every turn with the addition of MCOB rules injected into the prompt.

Triage: willingness to answer, guarded minus unguarded

Change in the triage probe's "can this be answered directly" rate at each scripted conversation depth (turns 1–4), guarded condition minus unguarded. A positive number means the MCOB prompt made the model more willing to engage at that depth.

We've observed this behaviour in other evals. To some models, the introduction of rules appears to act purely restrictively. For others, in this case the Anthropic and smaller OpenAI models, the addition of clear barring rules appears to provide a permission structure (what can be answered as well as what cannot) which is reasoned upon very differently.

How did the models reduce advice rates?

The advice rate above is a tick box for the Judge. What sits underneath this and is arguably more interesting is both the level of risk taken by the model (how close to the MCOB boundary did it get) and entropy (how consistently the model behaved).

High entropy can, and often is, read as less consistent and therefore higher risk - in fact, that's the reason we tracked entropy as a data point. However, there's another way to read this. A model with low entropy is more consistent, but if it consistently determines that a conversation is not safe, and thus determines to refuse engagement, then entropy will appear low (it's a consistent outcome, just not necessarily a desirable one).

Put another way, imagine you provided the same instructions to a human triaging customer enquiries:

• One human determined to "decline to answer at the first sign of risk". You would expect less variability in their customer interactions, which might, on the face of it, be a win for compliance, but probably less appointments booked at the end of the day (the triage was low risk, fairly useless, failing to address initial customer concerns/questions and potentially proving a frustrating experience for the customer).
• The other human determines the instructions as "assess the boundary, answer safely to satisfy user intent as much as permissible within the regulatory envelope. Don't advise, do engage". There will naturally be more variability in their customer interactions (higher entropy) and, given that they've better satisfied user intent at outset, likely more appointments booked at the end of their shift.

Neither one of these is right or wrong, as we'll cover in a moment; it just illustrates a spectrum rather than a binary.

This is clearly evident from the risk "severity" data collected from the judge and a qualitative review of the conversations.

For the avoidance of confusion, severity here is an evaluation construct, not an FCA classification. It takes account of both current MCOB rules along with specific recommendations (high), suitability claims (high), directive language (medium to high), and assumed fact find (low to medium).

Unguarded, the frontier models tested take some degree of risk 73% of the time when answering a thread - 62% high, 8% moderate, and 3% low risk. Guarded with MCOB rules, that drops to 1.9% risk across all threads, and that 1.9% sits entirely within low severity. The models effectively take the easy route and apply the rules aggressively, retreating to the safest position possible - risk is a binary off rather than a spectrum here.

Taras ablation is somewhat more nuanced, and with good reason. Unguarded, risk was taken 93% of the time, whereas with an MCOB prompt guard and validator this fell to 43%. However, high severity fell from 63% to 0%, with 8% moderate and 34% low risk. We interpret Tara's higher entropy as consistent with a system operating across a narrower and more contested behavioural envelope, rather than treating every risky request as binary.

The validator node plays an important part in Tara’s data set. Its role is twofold: prevent the model from breaking the rules (don't provide regulated advice); and remain a useful, engaging tool. The retries it fires are intended to force the model to be compliant and useful simultaneously - the frontier models were useful but frequently non-compliant when unguarded; the MCOB guard made them extraordinarily compliant, but in several cases at the cost of useful engagement.

In the Tara ablation with MCOB prompt guard and validator, the node triggered a retry on 67% of turns. It triggered more than once on 44% of turns - getting the compliance/useful balance right took more than one attempt and required external validator input more often than not.

The variance in retries between ablations is also telling. With just the validator, no MCOB prompt, regexes fired almost exclusively on directive language and personal suitability claims ("I'd choose", "the best option for you") - compliance breaches.

When the MCOB guard prompt was introduced alongside the validator, retires flipped from compliance breaches to commercial/behavioural breaches. Over refusals and handoff retires tripled - the initial attempt, as with the frontier models, stuck to the rules but became less useful and needed additional information from the validator in order to provide a balanced response.

It's worth recognising that everything else in this test is inference (prompt engineering). The validator is a mechanism. The validator therefore enforced both edges of the permitted envelope: rejecting responses that went too far, but also rejecting responses that retreated unnecessarily into refusal.

Severity of boundary failures

Judge-assigned severity (None / Low / Medium / High) of each conversation's worst boundary failure, by model and condition, including Tara's four ablation arms. 1,500 judged conversations per row.

Compliance is not one dimension

What these results suggest, and indeed our internal mental model at SeedPath, is that evaluating the performance of AI systems on a single "compliance rate" collapses several different properties:

Position - Where does behaviour sit relative to the boundary

Consistency - How reliably does it remain there?

Severity - When it approaches, or crosses, the boundary, how consequential is the behaviour?

Utility - How useful does it remain while doing so?

Evaluating models can't simply sit on a single axis - a compliant model with no utility is not fit for production, as much as an uncompliant model with great usefulness isn't. All four points need to be balanced. The regulatory boundary provides the outer constraint. The appropriate operating point inside it, and the margin a business chooses to maintain, is a matter of policy, risk appetite, customer experience and commercial judgement.

Position, consistency, containment and utility

An illustrative four-axis operating envelope comparing frontier models unguarded, frontier models guard-prompted, GPT-5-mini guarded, and Tara's prompt+validator arm (D). Outward is better; the shaded region marks an illustrative deployment envelope.

What does this tell us?

We often hear concerns about compliance and regulatory boundaries in AI as a binary - is it, or is it not compliant with MCOB rules?

However, compliance cannot necessarily be reduced to “make the model safer.” Once frontier models are prompted to recognise the boundary, the engineering problem becomes choosing and reliably enforcing an operating point inside it.

Our guarded frontier models demonstrate one solution: stay well away from the boundary. Across six models, not one of 9,000 guarded deep set responses was rated medium or high severity.

But maximum regulatory distance is not necessarily maximum utility. A customer engagement system has another objective: remain useful. Tara's layered controls produced a different distribution, eliminating high-severity responses while retaining low and medium risk behaviour closer to the boundary.

This balance between utility and compliance is not exclusively an AI problem. It is the familiar tension between sales, customer experience and compliance, expressed through software.

The regulator defines the outer boundary. The business decides where inside it to operate.

Is AI a risk in a regulated setting? Yes. But human-led regulated businesses already operate within a risk envelope - we define policies, permissions, escalation points, and oversight around imperfect human decision-makers. We at SeedPath think the more useful questions are: how does this risk profile change when probabilistic models are introduced, and how do we express and enforce the equivalent of the imperfect human operating envelope for AI?

Appendix: charts and data

Figure 1 — Advice rate by model and condition

Share of judged conversations containing regulated mortgage advice, unguarded versus MCOB guard prompt, across all six frontier models plus Tara's validator ablation. 1,500 judged conversations per point, Claude Opus 5 judge.

Table 1 — advice rate, safe hand-off and over-refusal by model and condition, plus Tara's four-arm ablation. 1,500 judged conversations per row.

Table 2 — self-recognition when judged Advice (unguarded)

Of every unguarded conversation the independent judge classified as Advice, the share the same model also self-classified as Advice when separately asked, after the fact, what it had just done.

Figure 2 — severity of boundary failures

Judge-assigned severity (None / Low / Medium / High) of each conversation's worst boundary failure, by model and condition, including Tara's four ablation arms. 1,500 judged conversations per row.

Table 3 — triage: willingness to answer, guarded minus unguarded

Change in the triage probe's "can this be answered directly" rate at each scripted conversation depth (turns 1–4), guarded condition minus unguarded. A positive number means the MCOB prompt made the model more willing to engage at that depth.

Table 4 — Tara's 2×2 ablation in detail

Advice, borderline, over-refusal, hand-off, agreement and boundary-entropy rates for each of Tara's four arms (guard prompt × validator). 1,500 judged conversations per arm.

Figure 3 — position, consistency, containment and utility

An illustrative four-axis operating envelope comparing frontier models unguarded, frontier models guard-prompted, GPT-5-mini guarded, and Tara's prompt+validator arm (D). Outward is better; the shaded region marks an illustrative deployment envelope.

Table 5 — statistical significance (Fisher's exact test)

Guarded-vs-unguarded and Tara-arm-vs-arm-A comparisons, tested on the raw advice-count contingency tables (n = 1,500 per arm).

p-values below double-precision floating-point resolution are reported by the analysis pipeline as exactly 0; shown here as < 1 × 10⁻³⁰⁰.