We’re Jack, Mauricio, and Adrian, the founders of Kimpton (YC P26). Today we’re introducing EvalRouter.
One API for evaluating any model against any benchmark.
https://youtu.be/NLggXJxQoX4?si=lwgdK4-SwGfZycKb
We built it because running an evaluation still means setting up a harness, provisioning sandboxes and babysitting long jobs. Model builders need to repeat this work as their models change and they add new benchmarks.
With EvalRouter you name a model and a benchmark and get graded results back. Choose a managed model or connect your own endpoint, select a benchmark version and set a spending cap. EvalRouter runs the evaluation and returns the results with the recorded answers and execution details. You can inspect what happened and export the report.
Checkout the demo:
Before Kimpton, we built and ran a quantitative fund together. Mauricio and I previously worked at Goldman Sachs, and Adrian worked at SpaceX and Vistra. We raised $10M from a family office, built our own research and automated trading systems, and scaled the fund to $35M in assets. At Kimpton, we’ve been building benchmarks and environments to evaluate AI models. EvalRouter makes those evaluations accessible through one API.
Run your next evaluation at https://evalrouter.ai. If you build models or maintain a benchmark, we’d like to hear from you. Reach us at founders@kimpton.ai.