A team asks whether it should fine-tune a model on the company handbook. The handbook changes every month, and half the staff should not see half of it.
Before choosing the cleverer technique, it is worth describing the problem properly.
The short answer: use retrieval-augmented generation when answers need relevant, changeable source material. Consider fine-tuning when the model needs more consistent task behaviour, style or output patterns demonstrated by good training examples. Knowledge access and model behaviour are different design problems.
What does RAG change?
RAG retrieves relevant material and provides it as context at answer time. The underlying model need not be retrained for each document update. The system can show supporting sources, although those sources still need to be relevant and genuinely support the answer.
Its failure modes include retrieving the wrong passage, missing a qualification, using stale content or generating an unsupported conclusion. RAG is a pattern for grounding, not a guarantee of truth.
For implementation detail, start with building an AI knowledge base and the Microsoft RAG overview.
What does fine-tuning change?
Fine-tuning adjusts model parameters using training examples. Depending on the method and model, it can improve consistency on a task or teach desired response patterns. It requires suitable examples, evaluation and a deployment arrangement that supports it.
It is not a dependable substitute for a document retrieval and permission system. You should not assume that facts incorporated into model weights can be selectively retrieved, cited, corrected or hidden from a particular user.
Start by checking whether better instructions, examples in context or a simpler workflow solve the behaviour problem. Fine-tuning introduces another asset to maintain, not merely a button to press.
Compare the decision
| Need | Starting approach | Critical check |
|---|---|---|
| Frequently changing policies | RAG | Current sources and update propagation |
| Different document access by user | Permissioned retrieval | Access enforced before generation |
| Consistent task format | Instructions and validation first; consider fine-tuning | Held-out task performance |
| Specific writing or classification patterns | Consider fine-tuning with representative examples | Generalisation and regressions |
| Answers with inspectable evidence | Retrieval plus source presentation | Citation actually supports the claim |
| Both knowledge and consistent behaviour | Potentially combine approaches | Separate tests for each component |
This is a design guide. The right decision depends on the available models, data and task evidence, not a universal hierarchy of sophistication.
Three illustrative choices
Staff policy assistant: begin with permissioned retrieval over current policies. Test conflicts, effective dates and missing answers. A fine-tuned model alone is a poor place to maintain changing employment rules and role-specific access.
Support ticket classification: begin with instructions and labelled examples. If performance remains inconsistent and you have enough high-quality data, assess fine-tuning against a held-out test set. Preserve an escalation class for ambiguous cases.
Technical report drafting: retrieve approved evidence, then use a controlled output structure. Fine-tuning may be considered for recurring task behaviour, but it does not remove the requirement to verify factual claims.
Measure the component that is failing
If the right passage never reaches the model, fix retrieval or the source collection. If the passage is present but the answer ignores a constraint, examine instructions, model behaviour and validation. If reviewers disagree about the desired answer, clarify the task before training anything.
Separate development examples from evaluation examples. Test ordinary cases, edge cases and failures that matter to the business. Compare with a simple baseline, including the human effort required to review output.
A larger training set made of inconsistent examples may reinforce the inconsistency. Quality control on labels and expected outputs is real work.
Include ongoing cost and change
RAG brings indexing, retrieval, source maintenance and context-processing costs. Fine-tuning brings data preparation, training, evaluation and model maintenance. Either may require specialist support and retesting after changes.
Avoid quoting a universal cheaper option. Estimate the workload using your expected volume, update frequency, context size and review requirements. Revisit the estimate after a pilot.
Choose the technique that addresses the observed failure. If the problem is an out-of-date document, buying a more elaborate model is an unusually indirect way to fix it.
Sources and scope
Reviewed 1 November 2026. The examples and worksheets are my suggested working methods, not measured client results or official standards.
