On this page · 8 sections
Summary. Retrieval-augmented generation (RAG) is the right first choice for roughly 80% of enterprise LLM applications, because it lets you change source data without retraining and cite the documents behind an answer. Fine-tuning solves the other problem: it changes how a model behaves, not what it knows. In 2026 almost all practical fine-tuning uses LoRA or QLoRA, which freeze the base weights and train small adapters, cutting memory needs by more than 90% and making the old "1,000 examples minimum" rule obsolete, with 200 to 500 curated examples often enough for classification. The cost is small: a LoRA run on a 7 to 8 billion-parameter open model is $3 to $10 of GPU time, a 70B QLoRA job is $15 to $30, and a managed equivalent is about $25. The hard part is not the training; it is deciding whether to fine-tune at all, preparing the data, and evaluating the result. This is a capability eCorpIT builds for teams fine-tuning open models such as Llama, Qwen, DeepSeek, Kimi K3, and GLM.
The question we hear most from founders and heads of AI is some version of "should we fine-tune, or is RAG enough?" The honest answer is that most teams reach for fine-tuning too early, pay in complexity, and get a result a good retrieval pipeline would have matched. But there is a real set of problems where fine-tuning is the only tool that works. This article draws that line, prices the work at 2026 rates, and explains how we deliver it.
Fine-tune, RAG, prompt, or API: the decision
Four options sit on a ladder of cost and control. Start at the cheapest that solves your problem, and only climb when it fails.
| Approach | Best for | Data changes | Upfront effort | Ongoing cost |
|---|---|---|---|---|
| Prompt engineering | Quick behaviour changes, prototypes | Any time | Hours | API tokens |
| RAG | Knowledge that updates; answers needing citations | Update the index, no retrain | Days to weeks | Retrieval plus tokens |
| Fine-tuning (LoRA) | Fixed style, format, tone, narrow tasks | Requires a new run | Weeks; needs curated data | Cheap inference, own hosting |
| Full self-host plus fine-tune | Data control, high volume, deep adaptation | New run | High; infrastructure and ops | GPU hours |
The rule of thumb that holds in production: RAG expands what a model knows, fine-tuning changes how it behaves, and the best systems are hybrids that use RAG for what changes and fine-tuning for what should not. If your problem is "the model does not know our latest policy," that is RAG. If it is "the model will not consistently produce our output format or house tone," that is fine-tuning.
When fine-tuning actually wins
Four cases justify the extra work. The first is style, tone, and output structure that prompting cannot hold reliably across thousands of calls, such as a fixed JSON schema or a regulated document format. The second is narrow classification and extraction, where a small fine-tuned model beats a large general one on accuracy and cost. The third is distillation: fine-tuning a smaller open model on a frontier model's outputs to get most of the quality at a fraction of the latency and price. The fourth is compliance and domain language, where the model must adopt a specific vocabulary or refuse in a specific way every time. Outside these, RAG or better prompting is usually the cheaper answer, which is why our RAG knowledge-assistant service is where many engagements start.
The methods, briefly
LoRA (Low-Rank Adaptation) is the default in 2026. It freezes the original weights and injects small trainable matrices into each layer, so you train a few million parameters instead of billions. QLoRA adds 4-bit quantisation of the frozen base, which is why a 70B model fits on a single 80GB GPU and an 8B model trains on a consumer card, at a 1 to 3% quality cost versus full fine-tuning on most benchmarks. Supervised fine-tuning (SFT) on input-output pairs is the workhorse; preference tuning such as DPO aligns tone and refusals; continued pretraining adapts a model to a new domain corpus before task tuning. For most business problems, LoRA SFT on a few hundred to a few thousand clean examples is the whole job.
What it costs in 2026
Fine-tuning is no longer the expensive part of an LLM project; data preparation and evaluation are. Verified July 2026 GPU rates on RunPod, Lambda, and Vast.ai run about $1.19 per hour for an A100 80GB, $1.99 for an H100, and roughly $0.31 for an RTX 4090.
| Job | DIY GPU cost | Managed cost | Hardware |
|---|---|---|---|
| LoRA on 7 to 8B model | $3 to $10 | ~$25 | RTX 4090 or one A100, 2 to 4 hours |
| QLoRA on 70B model | $15 to $30 | Higher tier | Single A100 or H100 80GB |
| Managed LoRA SFT (Together, sub-16B) | n/a | ~$0.48 per M tokens | Provider-hosted |
A managed 7B QLoRA job with 50,000 examples runs around $24.58 on Together AI and $25.60 on Fireworks, versus $3 to $10 to run it yourself on rented hardware. In Indian terms the training bill is trivial, a few hundred rupees of GPU time; the real investment is the engineering to build a clean dataset and an evaluation harness that proves the fine-tuned model is actually better. Where inference volume is high, pairing a fine-tune with private LLM deployment or a self-hosted open model such as Kimi K3 is what turns a cheap training run into a durable cost advantage.
Which open model to fine-tune
The base model choice depends on the task and your hosting plan. Llama and Qwen have the widest tooling and adapter support, DeepSeek and GLM are strong on reasoning and code, and the largest open models such as Kimi K3 suit teams that need frontier-class quality in their own environment. A smaller 7 to 14B model is usually the right target for a fine-tune, because it is cheap to train, cheap to serve, and easy to iterate. If you are already weighing a migration off a paid API, our LLM migration and cost-optimisation service covers the base-model decision alongside the spend model.
How eCorpIT can help
eCorpIT is a Gurugram-based, senior-led engineering organisation, founded in 2021 and certified for CMMI Level 5, MSME, and ISO 27001:2022. We deliver open-model fine-tuning end to end: the fine-tune-versus-RAG decision, dataset design and cleaning, LoRA or QLoRA training, an evaluation harness that measures the result against your real tasks, and deployment on your cloud or on a hosted endpoint. We design these systems aligned with DPDP Act 2023 requirements so training data and outputs are handled lawfully. A typical engagement starts with a short scoping review that decides whether fine-tuning is even the right tool, so you do not pay to train a model you did not need. To scope a domain-adaptation project, contact us.
FAQ
References
_Last updated: 29 July 2026._
