The Best Model Is a Routing Decision
The best model isn't the smartest — it's whichever should get the next unit of work. Notes from NVIDIA on routing, local models, and the AI factory.
Latest insights on API testing, performance optimization, and software development best practices from the Speedscale team.
The best model isn't the smartest — it's whichever should get the next unit of work. Notes from NVIDIA on routing, local models, and the AI factory.
A production bug survived two confident fixes and green tests. Replaying the captured request exposed the missing state and proved the real fix.
Testing AI applications with invented traffic looks fine until real users arrive. Then come the retries, the fallback models, and the token bill.
AI pushed throughput up 59%, yet median delivery got worse. The bottleneck moved to validation. How replaying production traffic in CI closes the gap.
The trace was sampled out. I found the bug anyway — by filtering recorded traffic on the customer's email instead of a trace ID. Here's how to follow one request across four services with no trace IDs and no OpenTelemetry.
Metrics, logs, and traces were built for humans and cheap storage. AI inverts both assumptions, and the next maturity level is a deterministic replay sandbox.
A second run of our AI bug-fixing benchmark shows where captured traffic lifts agents toward 90%, why service maps barely help, and which bugs still fail.
Using 'production-similar' data in pre-production is a major security risk. Learn why traditional masking fails, where hidden PII hides, and how to fix it.
Logs, metrics, and traces are a lossy compression of production. Five things you can do with a traffic data lake that observability can't.