Consistency
Train for the repeated task
Fine-tuning gives the model repeated examples of the decisions, tone and boundaries required in the real workflow.
Model note / Tara / July 2026
We changed 2.68% of Tara. Her behaviour improved by 39 points.
Fine-tuning did not teach a general model a new world of facts. It made a small, concentrated adjustment to how the model behaves in one tightly defined job.
Network-wide shift2.68%
Matrices adapted224
Top-4 direction energy73%
Projections × layers7 × 32
Schedule-action matchHeld-out evaluation
Base Llama 3.1 8B52%
Fine-tuned Tara91%
01 / The short version
Fine-tuning changed what Tara does, not what she knows.
Llama 3.1 8B is a general-purpose language model. It can discuss thousands of subjects, but it was not built specifically to guide a mortgage lead towards an adviser appointment while staying inside a carefully defined role.
We fine-tuned it on Tara's task. Afterwards, its match to the correct scheduling action rose from 52% to 91% on our evaluation. Yet across the parts of the network we adapted, the overall weight shift was only 2.68%.
In plain English
The model did not need to be rebuilt. It needed a relatively thin behavioural adjustment: how to respond, when to ask, when to book and when not to progress.
02 / What we measured
A small change, concentrated in a few directions.
A LoRA fine-tune records the difference between the base model and the adapted model. That allowed us to inspect the change directly from the weights—without asking another AI model to judge what happened.
Overall shift2.68%
The size of the update relative to the original weights across all adapted matrices.
Parts inspected224
Seven projection types across each of the model's 32 transformer layers.
Top four directions73%
Most of the update was carried by four of the 16 available directions—not spread evenly everywhere.
Correct schedule-action match on the defined Tara evaluation.
+39 pts
What stayed largely intact
The foundation model
LoRA leaves the original model weights in place and adds a compact learned update. Embeddings, layer normalisation and the final language-model head were not adapted in this run.
This is one reason fine-tuning can specialise behaviour without retraining the entire model from scratch.
What appears to have shifted
How the model routes and acts
The largest average movement was in the model's MLP gate projections—the parts involved in controlling which internal channels become active. The key projections, involved in attention, moved least.
That pattern is consistent with behavioural specialisation, although weights alone cannot tell us that one matrix learned a specific rule such as “offer this slot.” The evaluation is what measures the resulting behaviour.
03 / Why it matters
A specialist model can be smaller without being simplistic.
Tara does not need to be the best model at everything. She needs to be dependable at a narrow, inspectable job—and to hand regulated advice to a qualified adviser.
Control
Own the improvement cycle
We can evaluate, retrain and deploy against our own product requirements instead of waiting for a general model provider to change direction.
Efficiency
Use the model the job needs
A focused 8B model can deliver lower latency and inference cost than a much larger general model when the task is well defined.
Evidence
Measure behaviour separately
Weight analysis shows where and how much the model changed. Held-out evaluations test whether those changes actually improved the customer-facing task.
04 / Boundaries
What this study does not prove.
The claim is deliberately narrow. These are the boundaries the result should be read within.
- It is not a claim of general model superiority. The 52% to 91% result applies to a defined Tara scheduling evaluation.
- A small weight shift is not automatically a safe one. Safety and regulatory behaviour still require scenario testing, deterministic controls and human review.
- Weight location is not a transcript of learned reasoning. It supports interpretation; it does not reveal a neat rule stored in a particular layer.
- LLM-judged quality is useful but not ground truth. Judge-model comparisons need their methodology, human checks and limitations shown alongside the scores.
- This analysis covers one model and one fine-tune. The result should not be treated as universal evidence about LoRA or other models.
Tara became much better at her job without becoming a fundamentally different model.
That is the practical promise of specialist fine-tuning: preserve the broad capability of the foundation model, add a compact layer of task-specific behaviour, and test the resulting system against the decisions that matter.