Best LLMs for RAG Question Answering
Answering from retrieved passages with grounded, cited responses.
Typical request this page prices
- Quality floor: Top 40% of priced models by Intelligence Index (≥ 20.6).
- Costs use each provider’s current standard rates, with cached tokens at the cached-input rate where one is published. Speed is from Artificial Analysis where measured.
Lowest cost that meets the bar
Models that clear the quality floor, ordered by estimated cost for the request above.
| Model | Intelligence | Speed | Cheapest host | Input / output per 1M | Per 1,000 requests |
|---|---|---|---|---|---|
| Ling-3.0-flash-VL InclusionAI | 24.6 | 147tok/s | OpenRouter 3 providers | $0.021 / $0.062 cached $0.0042 | $0.172 |
| Ling-3.0-flash-Fin InclusionAI | 22.6 | 317tok/s | OpenRouter 3 providers | $0.042 / $0.123 cached $0.0084 | $0.344 |
| DeepSeek V4 Flash DeepSeek · 1.0M context | 34.3 | - | Deep Infra 19 providers | $0.060 / $0.180 cached $0.015 | $0.498 |
| GLM-5.3-Flash Z AI | 41.8 | 59tok/s | Pareto Inference 17 providers | $0.060 / $0.200 cached $0.012 | $0.503 |
| MiMo-V2.5 Xiaomi | 25.2 | - | GMI Cloud 7 providers | $0.119 / $0.238 cached $0.0024 | $0.884 |
| GPT-6 Luna OpenAI | 38.1 | 140tok/s | GMI Cloud 5 providers | $0.100 / $0.500 cached $0.010 | $0.906 |
| Claude Haiku 5.5 Anthropic | 43.4 | 238tok/s | DigitalOcean 3 providers | $0.100 / $0.500 cached $0.010 | $0.906 |
| MiMo-V2.6-Flash Xiaomi | 37.9 | 48tok/s | Deep Infra 7 providers | $0.140 / $0.280 cached $0.0028 | $1.04 |
Highest-scoring options
Ordered by Artificial Analysis Intelligence Index, with the cost of the same request.
| Model | Intelligence | Speed | Cheapest host | Input / output per 1M | Per 1,000 requests |
|---|---|---|---|---|---|
| Claude Opus 5.5 Anthropic | 57.6 | 97tok/s | Deep Infra 6 providers | $4.00 / $20.00 cached $0.200 | $35.92 |
| Claude Sonnet 5.5 Anthropic | 56 | 135tok/s | Anthropic 6 providers | $2.00 / $10.00 cached $0.100 | $17.96 |
| Claude Fable 5.1 Anthropic | 53.4 | 72tok/s | DigitalOcean 4 providers | $10.00 / $50.00 cached $0.250 | $89.40 |
| GPT-6 Astra OpenAI | 52.7 | 63tok/s | Velokey 7 providers | $3.00 / $15.00 cached $0.300 | $27.18 |
| GPT-6.1 Sol OpenAI | 51.8 | 65tok/s | Microsoft Azure 6 providers | $2.00 / $10.00 cached $0.100 | $17.96 |
| Claude Opus 5 Anthropic | 50.8 | - | Velokey 7 providers | $2.00 / $10.00 cached $0.200 | $18.12 |
| Claude Fable 5 Anthropic · 1.0M context | 49.6 | - | Velokey 6 providers | $4.00 / $20.00 cached $0.400 | $36.24 |
| Muse Spark 1.3 Meta | 48.1 | 115tok/s | OpenRouter 2 providers | $1.25 / $4.25 cached $0.150 | $10.36 |
Compare every model and provider on the LLM API pricing table.
How to choose an LLM for rag question answering
Retrieval-augmented generation sends the user’s question together with several retrieved passages, then asks for an answer grounded in those passages. Retrieved context makes the input several times larger than the output, and it changes on every query, so only the system prompt is cacheable.
The generation model needs to stay faithful to the passages and cite them, which favours models above the median on general capability. The floor here is set at the 60th percentile of priced models.
Embedding and reranking are separate calls with their own pricing and are not included in this estimate. For the self-hosted side of a RAG stack — embedding servers and vector search on your own GPUs — see the GPU RAG page linked below.
Frequently Asked Questions
Does this cost include embeddings?
No. The estimate covers the generation call only. Embedding the query is usually a small fraction of the total, while embedding the corpus is a one-time cost proportional to its size.
Should I retrieve more passages to improve answers?
More passages increase input cost linearly and can reduce accuracy when relevant text is buried among irrelevant text. Reranking a larger candidate set down to a few passages is often cheaper and more accurate than sending everything.
Is a long context window a substitute for retrieval?
For small corpora it can be, especially with prompt caching on a fixed document set. For large or frequently changing corpora, retrieval keeps per-query input small and the cost predictable.