LLM Tokens
LLM tokens (not to be confused with design-tokens), the unit of measurement for LLMs. One token ≈ ¾ of an English word (less in French). LLMs read and write in tokens, not words, not characters. Billing is always in tokens.
Strengths
- Universal, all LLMs use tokens, even if ratios vary
Limitations
- Token count varies by language, French consumes more tokens than English
Best for
- Anyone using an LLM API and wanting to control costs