Fastest inference anywhere: open models on-prem, hosted or on-device
Hi, we're Sanchit and Shubham, co-founders of RunAnywhere (W26), and we're launching Wally: our inference stack for open frontier models, hosted on our GPUs or deployed on yours.
TL;DR: Wally serves GLM-5.3 Flash at 380 tokens per second with a 274 ms first token, faster than any endpoint on Artificial Analysis (Sep 28, 2026). New models land on day zero: Xiaomi's MiMo-V2.6-Pro was ready to serve about 8 hours after its weights dropped, and runs at 3x the speed of Xiaomi's own API (Artificial Analysis, Sep 21, 2026). The same stack deploys on-prem/BYOC at the same speed. Sign up and get $5 in free credits.
The problem
Frontier models are slow and expensive, and you feel it most in products that run a model in a loop. A voice agent has a few hundred milliseconds before a turn feels broken. A coding agent waits for the first token on every tool call, dozens of times per task.
How Wally works
Wally is our inference stack for open frontier models: our own kernels, speculative decoding, and prefill kept off the decode path so a long prompt never stalls someone else's stream. We tune it per model and per traffic pattern. It runs in two places.
Hosted by us. An OpenAI-compatible endpoint on our own GPUs, so anything that takes a base URL and a key works unchanged. Standard per-token pricing. Live today:
- GLM-5.3 Flash: 380 tok/s
- DeepSeek V4.1 Flash: 615 tok/s
- Qwen3.8-27B: 485 tok/s
- MiMo-V2.6-Pro: 320 tok/s
- GLM-5.3 Max: 790 tok/s
Deployed for you. The same stack on GPUs you own, in your cloud account, or on a dedicated instance we run. Same speed, your own models and fine-tunes, your traffic stays in your environment, and at token-heavy volumes it costs less than paying per token.
New models on day zero
MiMo-V2.6-Pro, the top open-weights model on Artificial Analysis (Sep 21, 2026), was ready and running 3x faster than anyone else in 8 hours after Xiaomi published the weights.
The numbers
Artificial Analysis lists 19 providers serving the same GLM-5.3 Flash weights (Sep 24, 2026), and Wally is 8.6x faster than Z.ai. Same model; the only difference is how it's served. Ours: 380 tok/s and a 274 ms first token, measured the same way.
Beyond the benchmark
Benchmarks don't always match reality, so we ran a real prompt on four providers, three times: GLM-5.3 Flash in the same OpenCode harness on Wally, Nebius, Fireworks and Z.ai, all building a game at once.
The same model reasons more or less on the same prompt from run to run, on every provider, so finish times look close even when the work isn't. Tokens per second of wall clock is the stable number: Wally 213, Nebius 149, Fireworks 68, Z.ai 47, averaged across the runs.
Get started
Paste this into your coding agent:
set up https://runanywhere.ai/SKILL.md
Works with Claude Code, Claude Desktop and OpenCode. $5 free on signup, no card. Standard per-token pricing after that.
Why we built it
We spent nine months making open models fast on phones and laptops, with our own inference engines for Apple and Qualcomm chips (runanywhere-sdks, 10K+ GitHub stars). Our users kept needing models too big for the device, and nobody served open frontier models fast enough for a real-time loop. None of our work was specific to phones, so we pointed it at data-center GPUs.
Our ask
- If you run a voice or agent product and latency or the inference bill hurts, email founders@runanywhere.ai and we'll make it faster.
- If a customer has asked you for on-prem, intro us. We deploy on their GPUs.
- Try it and tell us what broke: runwally.com
RunAnywhere: The default way to run On-Device AI at scale
