Skip to content

Build a custom router

This workflow is about turning an idea into a router that works with the CLI.

Start from the notebook

The most complete, hands-on guide lives in the notebook: - Custom routers (notebook + plugin)

Minimal steps (plugin)

  1. Create a plugin folder:
custom_routers/my_router/
  __init__.py
  router.py

  1. Implement route_single and route_batch (see Routing output contract).

  2. Verify discovery:

llmrouter list-routers
  1. Run routing:
llmrouter infer --router my_router --config configs/model_config_test/smallest_llm.yaml --query "Hello" --route-only

Next