Choosing an AI model for a production app
A requirements-first framework for choosing models, providers, routing, evaluation, privacy controls, and fallbacks.
Start with the job
Define the input, expected output, acceptable latency, error cost, languages, data sensitivity and monthly volume before comparing providers. A fast small model can outperform an expensive frontier model on a constrained workflow.
Test the complete system
Retrieval quality, prompts, tools, schemas, retries and human review often matter as much as the base model. Test the exact system with production-like data and adversarial cases.
- Create a versioned evaluation set.
- Measure quality, p95 latency and total cost.
- Add timeouts, fallbacks and provider health checks.
- Log model versions without storing unnecessary sensitive content.
Keep an exit route
Use a thin provider adapter, avoid provider-specific assumptions in core business logic, and keep prompts and evaluation data portable. Model routing lets you change providers without rebuilding the product.