--- title: "SPECTRA — AI-search perception platform | Harshith Nayaka L" description: "AI-search perception platform: what an AI actually understands about a site, which facts disappear at which stage, and deterministic scoring backed by evidence." canonical: "https://harshith-nayaka-l-portfolio.vercel.app/work/spectra" last-updated: "2026-09-16T02:41:15Z" author: "Harshith Nayaka L" content-type: "text/markdown" html-version: "https://harshith-nayaka-l-portfolio.vercel.app/work/spectra" --- # SPECTRA — AI-search perception platform > Answers a narrower question than an SEO audit: what does an AI system actually understand about this site, what evidence supports that, and at which stage does a fact disappear? Case study by Harshith Nayaka L, AI Engineer (Full-Stack), Bengaluru, India. Canonical page: https://harshith-nayaka-l-portfolio.vercel.app/work/spectra - **Role:** Sole architect and engineer - **Stack:** Rust crawler, Bun/Hono API, React report - **Scoring:** spectra-v0.1, deterministic and versioned - **Status:** Built; deployment in progress ## The problem Search visibility is a pipeline, not a page score. A fact can sit in the HTML and still vanish on the way to an answer — dropped by the crawler, lost in extraction, flattened in the semantic representation, misread by the model, or never retrieved. By the time you see the output, you cannot tell which of those happened. The tools on either side of this are unsatisfying. Conventional SEO tools inspect technical health and rankings, which is a different question. Generic AI-visibility checkers ask a model to grade a site and hand back a number nobody can audit — the score is the model's opinion, and there is no way to check its work. ## What I built SPECTRA keeps the evidence through every stage and measures deterministically on top of it. A Rust crawler does the hostile network work and emits structured evidence; TypeScript validates that at the process boundary; Gemini produces typed observations; and a separate scoring engine computes the numbers from registered checks only. The load-bearing rule is that the model never produces a score. It classifies the site and emits observations against a schema, and those become entities, relationships and claims that each reference the evidence they came from. Scoring consumes measured checks, never prose and never a model-authored number. That is what makes a result arguable rather than something you either believe or don't. Classification runs first and decides what is even worth checking. The system establishes the primary entity, archetype and purpose before selecting evaluation dimensions, so a restaurant is not marked down against checks meant for a B2B SaaS site. Irrelevant checks come back N/A and contribute to neither the earned points nor the denominator. ## Pipeline - **Crawl:** Target URL (Validated and normalized) → DNS / IP safety (Rejects loopback, private, link-local, metadata) → Bounded fetch (Rust; 20 pages, depth 2, 2 MiB, 10s) - **Evidence:** Extraction (Metadata, headings, JSON-LD, links, fingerprints) → Normalization (De-noise, keep evidence references) → Schema validation (Crawler output untrusted until validated) - **Interpret:** Classification (Entity, archetype, purpose) → Typed observations (Schema-constrained; never scores) → Semantic graph (Entities, relationships, claims → evidence IDs) - **Retrieve:** Question generation (From evidence-backed claims) → Direct evaluation (What the model understands unaided) → Grounded retrieval (Optional; stored separately) - **Measure:** Dimension registry (Only registered checks can score) → spectra-v0.1 (Weighted checks over a visible denominator) → Evidence-backed report (Survival stages, issues, repairs) ## The judgment calls **The model observes; it never scores** Gemini classifies the site and emits observations against a response schema. Those are validated before anything downstream sees them, and the scoring engine accepts only typed checks from a controlled registry. An observation the registry does not know about cannot move the number. This is the difference between a score you can argue with and a score you can only accept. **Every claim records where it died** Each important claim carries a survival record across source, crawler, extraction, semantic graph, model understanding and retrieval — including the first stage that failed and why. That is the actual product: not 'your score is 62' but 'this fact is in your HTML and does not survive extraction'. **Partial results stay useful** A crawler failure is fatal, because without evidence there is nothing to measure. A model or grounding failure is not: the audit keeps the pages, metadata, structured data and every deterministic metric, and marks semantic evaluation unavailable. Without a Gemini key at all, the deterministic half still runs. Degrading to a blank page because one provider was down would have been the easy build and the useless one. **The crawler is the security boundary, so it is Rust and it is paranoid** It resolves and validates every destination and every redirect hop, rejecting non-HTTP schemes, embedded credentials, non-public addresses, oversized bodies and unsupported content types. Raw HTML never reaches the model. The defaults — 10s timeout, 2 MiB per response, 5 redirects, 20 pages, depth 2, concurrency 4 — are defense in depth, not a claim that arbitrary remote content is safe. **Audits are immutable snapshots** An audit is sealed on completion with its scoring version, so a re-scan is a second snapshot rather than an overwrite. Comparison aligns claims by normalized subject/predicate/object and evidence fingerprints and reports the deltas. Scores computed under different versions stay distinguishable instead of being silently mixed. ## What it changed **What it produces:** An evidence-linked report: the site's interpreted identity and purpose, an entity map, per-claim information-survival stages, retrievability and paraphrase stability, structured-evidence coverage, positioning bugs and the repairs for them — each traceable back to the source it came from. **Why the scoring is deliberately dull:** spectra-v0.1 is weighted binary and ratio checks over a visible denominator. Each metric reports its version, calculation, applied and N/A checks, earned points and limitations. The initial weights are stated as engineering assumptions to be calibrated against labeled fixtures — not presented as science. **Honest scope:** V1 has one production model provider, and search-grounded retrieval only runs where the configured Gemini API and model support it. The fixtures are test evidence, labeled as such, not dashboard metrics. Deployment is in progress; the live link and interface capture go here once it is up. ## Built with Rust, Bun + Hono, React + Vite, Gemini (schema-constrained), TypeScript monorepo, PostgreSQL (production model), Deterministic scoring engine ## Links - [View on GitHub](https://github.com/HarshithNayakaL/spectra)