EvalRouter by Kimpton: One API for AI evaluation
Evaluate any model against any benchmark
We’re Jack, Mauricio, and Adrian, the founders of Kimpton (YC P26). Today we’re introducing EvalRouter.
One API for evaluating any model against any benchmark.
https://youtu.be/NLggXJxQoX4?si=lwgdK4-SwGfZycKb
We built it because running an evaluation still means setting up a harness, provisioning sandboxes and babysitting long jobs. Model builders need to repeat this work as their models change and they add new benchmarks.
With EvalRouter you name a model and a benchmark and get graded results back. Choose a managed model or connect your own endpoint, select a benchmark version and set a spending cap. EvalRouter runs the evaluation and returns the results with the recorded answers and execution details. You can inspect what happened and export the report.
Checkout the demo:
Before Kimpton, we built and ran a quantitative fund together. Mauricio and I previously worked at Goldman Sachs, and Adrian worked at SpaceX and Vistra. We raised $10M from a family office, built our own research and automated trading systems, and scaled the fund to $35M in assets. At Kimpton, we’ve been building benchmarks and environments to evaluate AI models. EvalRouter makes those evaluations accessible through one API.
Run your next evaluation at https://evalrouter.ai. If you build models or maintain a benchmark, we’d like to hear from you. Reach us at founders@kimpton.ai.