---
title: "CraftConnect — Gen AI Exchange Hackathon 2025 | Harshith Nayaka L"
description: "Multi-modal assistant letting artisans run an online storefront by talking and showing instead of typing. Gen AI Exchange Hackathon 2025 semi-finalist."
canonical: "https://harshith-nayaka-l-portfolio.vercel.app/work/craftconnect"
last-updated: "2026-08-22T04:29:48Z"
author: "Harshith Nayaka L"
content-type: "text/markdown"
html-version: "https://harshith-nayaka-l-portfolio.vercel.app/work/craftconnect"
---
# CraftConnect — Gen AI Exchange Hackathon 2025

> Let an artisan stand up an online storefront by talking and showing a product, instead of typing forms in a language that isn't theirs.

Case study by Harshith Nayaka L, AI Engineer (Full-Stack), Bengaluru, India.
Canonical page: https://harshith-nayaka-l-portfolio.vercel.app/work/craftconnect

- **Role:** Product lead, full frontend, co-built backend
- **Context:** Gen AI Exchange Hackathon 2025
- **Scale:** 270,000+ developers nationally
- **Status:** Built & deployed on Google Cloud

## The problem

Most artisans don't lose sales because their craft isn't good enough. They lose them because getting a product online means typing English product descriptions, setting prices, writing tags, and fighting a form-heavy interface that assumes you already know e-commerce.

That barrier is the whole problem. The skill is in the hands; the friction is in the keyboard.

## What I built

CraftConnect removes the keyboard. An artisan photographs a product and describes it out loud in their own language. From that, the system produces a complete, structured listing: a title, a written description, suggested categories and tags, and a storefront page, ready to publish.

It's multi-modal by necessity, not for show. The photo carries information the voice doesn't (material, colour, form), and the voice carries information the photo can't (story, intended use, price intent). The pipeline fuses both into one validated listing.

I led the product, built the entire frontend, co-built the backend, and owned the decisions that actually mattered: which AI services to use where, and what the end-to-end user journey should feel like.

## Pipeline

- **Capture:** Voice input (Artisan's own language) → Product photo (One or more images)
- **Understand:** Speech-to-text (Transcribe + detect language) → Vision analysis (Gemini reads the product)
- **Compose:** Fuse signals (Voice intent + visual attributes) → Listing generation (Gemini, schema-constrained)
- **Verify:** Validate fields (Title, price, tags present & sane) → Localize (Buyer-facing language)
- **Publish:** Storefront page (Ready to go live)

## The judgment calls

**Voice and vision are treated as two witnesses, not one**

The photo and the spoken description are analyzed separately, then reconciled. When they agree, confidence is high. When they conflict (the voice says 'silk' but the image reads cotton), the system surfaces it rather than silently guessing. That's a deliberate reliability choice, not an accident of the model.

**Generation is schema-constrained, not free-text**

The model isn't asked to 'write a listing'. It's asked to fill a defined structure: title, description, category, tags, price band. Constraining the output shape is what makes it usable by the rest of the app instead of being a paragraph someone still has to parse.

**Language is decoupled from logic**

The artisan speaks one language; the buyer may read another. Localization happens as its own step at the end, so the understanding pipeline never has to care what language it started in. One pipeline, many markets.

**AI service selection was a judgment call, not a default**

I chose where to spend model capability and where a cheaper, narrower service was enough. Speech-to-text, multimodal reasoning, and generation are different jobs with different cost and latency profiles, and they were wired accordingly.

## What it changed

**What changed:** A task that previously required literacy in English, e-commerce, and form-filling collapses into 'show it and say it'. The artisan's expertise stays in the craft, not the keyboard.

**Recognition:** Built and deployed for the Gen AI Exchange Hackathon 2025, a national event with 270,000+ developers, and reached the semi-finals.

**Honest scope:** This was a hackathon build, not a production marketplace. The value it proves is the interaction model and the multi-modal pipeline behind it, both of which hold up.

## Built with

Gemini (multimodal), Google Cloud Speech-to-Text, Google Cloud, React, Node.js, Structured / schema-constrained output

## Links

- [View on GitHub](https://github.com/HarshithNayakaL/craftconnect)