Question 1

What is the cost of OpenAI text-embedding-3-small?

Accepted Answer

OpenAI text-embedding-3-small costs $0.02 per million tokens as of 2025 — down 90% from the original ada-002 model at $0.10/million. For a typical RAG pipeline processing 10 million tokens per month, this equals $0.20/month. The text-embedding-3-large model costs $0.13 per million tokens and produces higher quality embeddings for complex semantic tasks.

Question 2

How many tokens are in a typical document chunk for RAG?

Accepted Answer

Standard RAG chunking uses 256–512 tokens per chunk for most use cases. A 1-page document typically generates 3–5 chunks at 512 tokens each. Best practice is to chunk at natural boundaries (paragraphs, sections) rather than fixed character counts. Overlap of 10–20% between chunks (50–100 tokens) improves retrieval quality without significantly increasing cost.

Question 3

Should I use OpenAI embeddings or run my own?

Accepted Answer

For most applications under 100M tokens/month, OpenAI text-embedding-3-small is the most cost-effective option at $0.02/million tokens. Above 100M tokens/month, self-hosted models like sentence-transformers (e5-large, bge-large) become cost-competitive on GPU infrastructure at roughly $0.005/million tokens fully loaded. Quality depends heavily on domain specificity — general models work well for English text.

Question 4

Do I need to re-embed documents when my knowledge base changes?

Accepted Answer

Yes, but strategically. Only re-embed documents that actually changed — use content hash tracking to detect changes. For static knowledge bases, a monthly re-embedding pass catches drift from model updates. For dynamic content (news, product catalogs), incremental embedding of new/changed documents daily keeps costs 80–95% lower than full re-embedding. Cohere's embed-v3 supports incremental updates natively.

Question 5

What is the embedding cost for a 1 million document corpus?

Accepted Answer

A 1M document corpus at 512 tokens/chunk = 512M tokens. At OpenAI text-embedding-3-small pricing ($0.02/million), the one-time embedding cost is $10.24. Monthly query costs depend on search volume: at 100K queries/day at 100 tokens each = 300M tokens/month = $6/month. Total first-year cost: ~$82 — making embeddings one of the most cost-efficient AI infrastructure components.

AI Embedding Cost Calculator — OpenAI, Cohere & Open-Source Pricing

Frequently Asked Questions

From the Blog