Groq
Inference provider running open-weight models (Llama, Mixtral, Qwen) on custom silicon called an LPU rather than the usual GPUs. The concrete result: answers streaming at 300 to 1000 tokens per second where a classic provider gives you 30 to 80. The API speaks the same dialect as OpenAI, so wiring Groq in means swapping a URL and a key in existing code. Free tier with no credit card, roughly 30 requests per minute, enough to prototype on.
Strengths
- Outlier response speed (300 to 1000 tokens per second) that changes how the product feels
- OpenAI-compatible API: plugging in or out means swapping a URL and a key
- Free tier with no credit card, usable for a real prototype and not just a demo
Limitations
- Catalog limited to open-weight models: no Claude, no GPT, so not a replacement for long reasoning
- Free tier quotas hit quickly as soon as several users hit it at the same time
- Single-vendor dependency for the speed advantage: plan an exit if the API slows down
Best for
- Prototyping an agent or chat where the wait is visible on screen, with no infrastructure budget
- Loops chaining many small model calls where accumulated latency stings