vLLM Semantic Router

01

End to end

One project, the whole distance.

01Research NeurIPS 2026 · two posters Plus SIGIR and EMNLP, with 10 research partners. 02Open source 6.4K GitHub stars 280+ contributors. 1.8M+ downloads. 03Models 121 open models Decision 3.0 debuted at #1 on the Jev Decision Index. 04Product Duo System One + System Two Decide in a blink. Reason in depth. 05Industry 14 companies building with us UBS, Bayer, Dell, Red Hat, Microsoft, Hugging Face.

02

Research

Research that ships.

All papers →

Reason only when needed

When to Reason · NeurIPS 2025 MLForSys

+10.2pp MMLU-Pro accuracy

−47.1% Latency

−48.5% Tokens

Models collaborate, the system wins

Opus 4.8 + Gemini 3.1 Pro + GPT-5.5 through vLLM-SR

96.0+1.7 GPQA-Diamond · vs. best single model

92.6+1.9 LiveCodeBench · vs. best single model

NeurIPS 2026Poster

The 1/W Law: Context Length is the Dominant Energy Lever in LLM Inference Fleets

NeurIPS 2026Poster

MOSAIC: Scaling Long-Horizon Language Agents via Multi-Scale Adaptive Inference Control

EMNLP 2026GroundLM Workshop

Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents

SIGIR 2026Industry Track

Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems

NeurIPS 2025MLForSys Workshop

When to Reason: Semantic Router for vLLM

Research partners 10

MBZUAI
McGill University
UC Berkeley
University of Chicago
New York University
UMass Boston
UIC
National Taiwan University
IBM Research
KR Labs

03

Open source

Built in the open, inside the vLLM project.

GitHub ↗

6.4K GitHub stars

280+ Contributors 5.6× in 2026

2.5K+ Merged PRs

1.8M+ Downloads

GitHub stars · releases 2025.08 — 2026.10

2K4K6K LaunchIrisAthenaThemisHermes

Merged PRs per month Record · 476 in Oct

A

S

O

N

D

J

F

M

A

M

J

A

457 S

476 O

Downloads All time

1.85M

934KHugging Face
524KPyPI
393KContainers

982Commits in v0.4 Hermes
107First-time contributors in v0.4
7Open workgroups
12Organizations maintaining

As of Nov 6, 2026. GitHub, Hugging Face, PyPI, and GHCR.

04

Models

Open foundation decision models. Trained on AMD Instinct, served by vLLM.

Hugging Face ↗

121 Open models on Hugging Face

#1 Decision 3.0 · Jev Decision Index

#1 #2 #3 Hugging Face trending collections

Decision

Choices, yes/no, and scores, with a probability for every answer.

2026.10 · 0.6B — 27B Decision3.0 Text, images, and video in every size. #1 Jev Decision Index 2026.10 · 0.6B — 27B Decision2.0 64 questions about one request in 63 ms. #1 at 0.6B — 4B 2026.09 · 0.6B — 9B Decision1.0 Kai, Lex, Eos, Sol, Nox, and Lux. 6 models

Vela

Routing, safety, and exact text spans in one call. With KR Labs.

2026.10 · 0.3B — 27B Vela3.0 Name it. Vela finds it. Built on Decision 3.0 2026.10 · 0.3B — 9B Vela2.0 Routing, prompt attacks, PII, and claims in one pass. 0.99 attack AUC 2026.09 · 307M Vela1.0 Encoders with 32K multilingual context. 14 models

System One Auto

Small first. Big only when unsure.

Every requestRequest
Answers firstKai · 0.6B
Only if unsure
EscalatesVega · 27B

−54% Cost vs. Vega alone

−20% Latency vs. Vega alone

+16.45 Points vs. Kai alone

05

Product

From open models to a product you can call in one click.

app.vllm-sr.ai ↗

app.vllm-sr.ai System Duo Thinking, fast and slow. Decide in a blink. Reason in depth.

System One POST /v1/systemone

Typed decisions over text, images, and video, with a probability for every option.

d3-ultra · d3-flash · vela3-turbo · vela3-flash

System Two POST /v1/chat/completions

One name, vllm-sr/auto, in front of any model. Each request gets the model and effort it needs.

Open or closed · any provider

System One · one call, a probability for every option

curl -s https://app.vllm-sr.ai/v1/systemone \
  -H "Authorization: Bearer $VLLM_SR_API_KEY" \
  -d '{"model": "vllm-sr/d3-ultra",
       "state": "Our app has returned 500 errors since the 09:00 deploy.",
       "questions": {"route": {"type": "choice",
         "instructions": "Which team should handle this ticket?",
         "criteria": {"billing": "Payments and refunds",
                      "engineering": "Outages and bugs"}}}}'