--- title: "Nova — Cost-tiered model routing | Harshith Nayaka L" description: "Chat app that scores every turn for difficulty and routes it across three model tiers, showing the lane, the reasoning and the cost on each answer." canonical: "https://harshith-nayaka-l-portfolio.vercel.app/work/nova-ai" last-updated: "2026-08-22T04:29:48Z" author: "Harshith Nayaka L" content-type: "text/markdown" html-version: "https://harshith-nayaka-l-portfolio.vercel.app/work/nova-ai" --- # Nova — Cost-tiered model routing > A chat app that reads every turn, scores how hard it is, and sends it to the smallest model that can carry it — stamping the lane, the reasoning and the cost onto every answer. Case study by Harshith Nayaka L, AI Engineer (Full-Stack), Bengaluru, India. Canonical page: https://harshith-nayaka-l-portfolio.vercel.app/work/nova-ai - **Type:** Deployed web app - **Roster:** gpt-oss-20b · qwen3.6-27b · gpt-oss-120b - **Routing:** Signal scoring + arbiter model - **Role:** Solo build ## The problem Every chat app sends one model everything. "Translate this line" and "find the race condition in this worker pool and prove it" land on the same endpoint, so you either overpay on the easy turns or underserve the hard ones. There is no third option when the roster is one model long. The obvious fix — a router that starts cheap and climbs — is the trap. Score from zero upward and almost every turn clears the bar for the fast lane, because most prompts trip two or three keywords at most. You end up with a router that technically works and quietly answers everything badly. ## What I built Nova scores each turn on four axes — reasoning, code, breadth and context — with deterministic keyword and pattern evidence that runs instantly and always runs. That produces a prior: a lane and a complexity reading between 0 and 100. Then a small arbiter model reads the same turn alongside the prior and returns a lane, a complexity number and a one-line rationale as JSON. It is allowed to overrule the arithmetic, because a scorer cannot tell that "explain CRDTs to someone who ships code, not papers" is a drafting job rather than a research one. Three lanes: gpt-oss-20b for lookups and rewrites, qwen3.6-27b for drafting and long context, gpt-oss-120b for reasoning, math and code review. Every answer is stamped with the lane it took, the complexity reading, the arbiter's rationale and what it cost, and the lane switch overrides routing entirely when you already know what you want. ## Pipeline - **Read:** Turn + history (What the user just sent) - **Score:** Four axes (reasoning · code · breadth · context) → Complexity prior (0-100, deterministic) - **Arbitrate:** Arbiter (gpt-oss-20b, JSON, 4s budget) → Floor rules (Pushback never routes below L2) - **Answer:** Lane model (20B, 27B or 120B) - **Show:** Stamped answer (Lane, complexity, rationale, cost) ## The judgment calls **The middle lane is the default, not the cheap one** A turn has to earn its way down to the fast lane by being demonstrably trivial, or up to the deep lane by being demonstrably hard. Climbing from zero would leave everything in the 20B model, which is the failure mode that makes routers feel worse than no router at all. Starting from the middle means the router only has to recognise the two extremes, which is the part it can actually do reliably. **The arbiter is allowed to be slow, wrong or absent** It gets a 4-second budget and a 120-token cap. If it times out, returns unparseable JSON, or names a lane that does not exist, the deterministic prior stands. If the routing endpoint itself fails, the browser falls back to its own copy of the scorer. Routing never blocks an answer — the worst case is a blunter decision, never a spinner. **Keyword evidence has diminishing returns** Each axis scores as 100 × (1 − decay^hits), so the first match already counts for most of the signal and the fifth adds almost nothing. A flat per-hit score reads real prompts as trivial, because people write two clauses, not a checklist of trigger words. **Disagreement is a routing signal** "That's wrong", "go deeper", "not what I asked" — a turn that pushes back on the previous answer never routes below the middle lane, whatever the keywords say. The last answer was already judged insufficient; sending the follow-up somewhere smaller is the one move guaranteed to be wrong. **One coordinator changes the answer** "Summarise this" is trivial. "Summarise this and list the open questions" is two asks wearing the same opener, so the presence of a coordinator disqualifies a turn from the trivial shortcut. Cheap tests like this catch the cases where a scorer would otherwise be confidently wrong. **Routing costs about as much as a greeting** The arbiter runs on the cheapest lane, capped at 120 tokens with temperature zero and JSON-mode enforced. Paying a 120B model to decide which model to use would defeat the entire exercise. ## What it changed **What changed:** Difficulty stopped being something the user has to declare. There is no model picker to get wrong, and no flat rate for turns that did not need it — but the decision is never hidden, because every answer shows the lane, the reading and the rationale that produced it. **What it took to be safe:** Three levels of fallback for one decision that is not allowed to fail: arbiter to prior, endpoint to browser-side scorer, automatic routing to a manual lane switch. The provider key stays in the serverless function throughout; the browser never holds a credential, and threads live in localStorage rather than on a server. **Honest scope:** A deployed personal app, not a production service. There is no evaluation set proving the routing beats always-using-the-large-model, and the lane assignments are judgement rather than measurement. The architecture is the claim here, not a benchmark. ## Built with gpt-oss-20b / 120b, qwen3.6-27b, Groq · Hugging Face routers, Vercel serverless, Streaming, Vanilla JS, no framework ## Links - [Live app](https://custom-gpt-silk.vercel.app/) - [View on GitHub](https://github.com/HarshithNayakaL/CUSTOM-GPT)