RAG, fine-tuning or automation: what do you need?

Choose by the gap you need to close

A team asking for a chatbot may really need a reliable search screen. A team asking to “train our own AI” may need current documents attached to each answer. Start by describing what fails today: finding the right information, producing a consistent result or completing a task across systems.

On a small screen, scroll the table sideways.

This table is a starting hypothesis. A short comparison using your own cases should decide the implementation. If a filtered list of source documents solves the problem, generated answers may add unnecessary work.

RAG supplies context at answer time

Retrieval-augmented generation retrieves relevant material and provides it to a model with the user’s question. The model does not have to learn those records through a new training run each time they change. Microsoft’s RAG design guide separates the document-processing pipeline from retrieval and answer generation, and recommends evaluating the stages.

For a buyer, the difficult work often sits around the model: deciding which document is authoritative, preserving access restrictions, extracting useful text, refreshing the index and showing the supporting source. A citation is only useful if it actually supports the statement and the reader may open it.

Ask for tests with an outdated policy, two conflicting versions, a scanned document and a question the collection cannot answer. Also test different users. A system that retrieves a restricted record and merely hides the link afterward has already made the wrong access decision.

Choose plain search when users need exact records and prefer to interpret them. Add generated answers when synthesis makes the task easier and the evidence can be checked. Plan what happens when there is no reliable answer.

Fine-tuning changes learned behavior

Fine-tuning uses training examples to adjust a model for a task. It can be relevant when a well-defined behavior remains inconsistent after a good baseline has been established. Examples include a specialized classification scheme or a recurring response format.

It is not a substitute for maintaining an up-to-date source system. If next week’s policy must appear in next week’s answer, design how that policy enters the application and how its version is checked.

OpenAI’s model-optimization guidance recommends establishing evaluations and iterating on prompts and representative examples. It also currently flags changes to its fine-tuning platform’s availability. Before selecting any provider, verify access, supported base models, retention terms, training cost and the path for model updates or retirement.

Keep training examples separate from the final test cases. Otherwise a good score may show familiarity with the examples rather than performance on new work. Confirm that you have the rights to use the data and that sensitive records are handled under an approved arrangement.

Automation owns the steps and actions

Search and generated text do not complete a business process by themselves. Creating a case, assigning it and updating its status require integrations, authorization, validation and a record of what changed.

A predictable process can use ordinary application code with a model at one step, such as extracting fields for review. An agent can choose its next tool or step where the task needs more flexibility. Anthropic’s architecture guidance distinguishes these patterns and notes the cost and latency tradeoffs of adding autonomy.

Prefer the amount of flexibility the task needs. Do not give a document-answering tool access to payment or deletion actions simply because a framework makes those connectors available. See the agent and automation buying guide for a closer look at action boundaries.

Example: a supplier onboarding desk

Imagine a procurement team that receives supplier documents, answers policy questions and creates onboarding records. This is a fictional example, not a client case study.

• Search: staff find a supplier’s submitted document and its current status.
• RAG: staff ask which policy requirements apply and receive an answer linked to the approved policy version.
• Model extraction: the application proposes fields from an uploaded document, with missing values marked for review.
• Workflow: a person approves the fields; the application validates them and creates the supplier record once.
• Possible later tuning: the team investigates a persistent classification error after collecting sufficient labelled examples and comparing simpler fixes.

These are separate acceptance decisions. A convincing policy answer does not prove that a record was created correctly, and successful field extraction does not prove eligibility to become a supplier.

Ask for a decision pilot with a useful finish

Give the delivery team an approved document sample, a small set of realistic questions and examples of the action you need completed. Agree the success measures, access boundaries and operating budget. Include cases that should produce no answer or no action.

Request a comparison with the simplest credible baseline, not just the most elaborate prototype. The output should explain which approach performed well, where it failed, what the next release includes and what it will cost to operate.

A useful first engagement can end with a decision to improve search, ship a bounded workflow or avoid a costly training programme. The value lies in resolving the business question with evidence you can inspect.