Fine-Tuning
Re-training an existing LLM on your own examples to change its weights, and therefore how it behaves: a style, a domain, a format. That is what sets it apart from RAG, which leaves the model untouched and simply hands it, at question time, information it does not have. Fine-tuning changes how the model answers, RAG changes what it knows in the moment. More powerful than prompt engineering for very targeted cases, also much more expensive.
Strengths
- Very precise results for very targeted use cases
Limitations
- High cost and rigidity, any data update requires a new fine-tuning run
Best for
- Understanding why fine-tuning is generally not the first solution to try