BrowserLLM

Run 100+ AI models entirely in your browser — no servers, no API keys, 100% private.

Features · How It Works · Tech Stack · Getting Started · Project Structure · Contributing · License

Think ChatGPT, but running on your hardware with zero cloud dependency.
All inference runs locally via WebGPU. Your conversations never leave your device.

---

✨ Features

🧠 AI, Fully Local 100+ models — Llama, Qwen, Phi, Gemma, Mistral, DeepSeek, SmolLM & more WebGPU-accelerated — Near-native GPU inference right in the browser Real-time streaming — Tokens stream as they're generated, rendered as Markdown	🔒 Privacy First Zero servers — No backend, no API calls, no data uploaded anywhere Works offline — PWA with service worker; fully functional without internet Local storage only — Chats saved in your browser, nothing touches the cloud
💬 Chat Experience Multi-thread — Create, switch, and manage multiple conversations Per-message stats — Tokens/sec, context usage, generation time for every response Markdown rendering — Code blocks with syntax highlighting, tables, lists Model name per message — See which model generated each response	⚡ Smart Model Management Background downloads — Download new models while chatting with the current one Browser cache — Models cached after first download, load in seconds next time Hardware detection — Auto-detects GPU, VRAM, WebGPU & shader-f16 support Default model — Set your preferred model, auto-selected when you open chat

---

🔧 How It Works

┌──────────────────────────────────────────────────────────┐
│                        Browser                           │
│                                                          │
│   ┌──────────────┐   postMessage   ┌─────────────────┐  │
│   │   React UI   │ ◄────────────► │   Web Worker    │  │
│   │  (main       │                 │  (MLC Engine)   │  │
│   │   thread)    │                 │                 │  │
│   └──────┬───────┘                 └────────┬────────┘  │
│          │                                  │           │
│          │ localStorage                     │ WebGPU    │
│          ▼                                  ▼           │
│   ┌──────────────┐                 ┌─────────────────┐  │
│   │  Chat        │                 │  Your GPU       │  │
│   │  History     │                 │  (VRAM)         │  │
│   └──────────────┘                 └─────────────────┘  │
│                                                          │
│   ┌──────────────────────────────────────────────────┐   │
│   │    Cache API — Model weights persisted locally    │   │
│   └──────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────┘

1. Download once — Quantized weights fetched from HuggingFace, stored in the browser Cache API
2. Web Worker isolation — MLC engine runs in a dedicated worker to keep the UI silky smooth
3. GPU inference — All matrix ops run on your GPU via WebGPU at near-native speed
4. Stream to UI — Tokens stream back in real-time via postMessage, rendered as rich Markdown
5. Persist locally — Chats saved to localStorage, restored on reload

---

🛠 Tech Stack

Layer	Technology
Framework	React 19 + TypeScript
Build	Vite
Styling	Tailwind CSS v4
LLM Runtime	@mlc-ai/web-llm via WebGPU
Routing	React Router v7
Animations	Framer Motion
Icons	Lucide React
Markdown	react-markdown + remark-gfm
PWA	Service Worker + Web App Manifest
Analytics	Vercel Analytics

---

🌐 Browser Support

Browser	Minimum Version	Status
Chrome	113+	✅ Supported
Edge	113+	✅ Supported
Safari	18.2+	✅ Supported
Firefox	—	❌ No WebGPU yet

Hardware: Small models (~0.5B) work with 2 GB VRAM. Larger models (7B+) need a dedicated GPU with 6–8 GB+ VRAM.

---

🚀 Getting Started

Prerequisites

• Node.js 18+
• A WebGPU-compatible browser

Quick Start

# Clone the repo
git clone https://github.com/GautamVhavle/BrowserLLM.git
cd BrowserLLM

# Install dependencies
npm install

# Start dev server
npm run dev

Open http://localhost:5173 and you're running.

Production Build

npm run build     # TypeScript check + Vite production build
npm run preview   # Preview the built app locally

All Scripts

Command	What it does
npm run dev	Dev server with hot reload
npm run build	Type-check → production build
npm run preview	Serve production build locally
npm run lint	Run ESLint

---

📁 Project Structure

BrowserAI/
├── public/
│   ├── favicon.svg              # App icon
│   ├── manifest.json            # PWA manifest
│   ├── sw.js                    # Service worker
│   ├── robots.txt               # Search engine crawl rules
│   └── sitemap.xml              # XML sitemap
│
├── src/
│   ├── main.tsx                 # Entry — React root + SW registration
│   ├── App.tsx                  # Route definitions
│   ├── index.css                # Global styles + Tailwind
│   │
│   ├── types/index.ts           # Shared interfaces (Message, ChatSession, etc.)
│   │
│   ├── hooks/
│   │   ├── useWebLLM.ts         # Core — model loading, inference, stats
│   │   ├── useChatManager.ts    # Chat state — wraps useWebLLM + localStorage
│   │   ├── useHardwareDetect.ts # GPU/VRAM/WebGPU detection
│   │   ├── useModelCache.ts     # Cache API introspection
│   │   └── useOnlineStatus.ts   # Online/offline detection
│   │
│   ├── lib/
│   │   ├── modelCatalog.ts      # 100+ model definitions with metadata
│   │   ├── models.ts            # Public API re-exports
│   │   ├── storage.ts           # localStorage CRUD
│   │   ├── constants.ts         # Landing page content data
│   │   └── animations.ts        # Framer Motion variants
│   │
│   ├── workers/
│   │   └── engine.worker.ts     # Web Worker — MLC engine off main thread
│   │
│   ├── components/
│   │   ├── chat/                # Chat UI (layout, messages, input, stats, sidebar)
│   │   ├── landing/             # Landing page sections (hero, features, FAQ, etc.)
│   │   └── ui/                  # Shared UI (star field, loading bar, indicators)
│   │
│   └── pages/
│       └── ModelsPage.tsx       # Full model catalog with filters + hardware compat
│
├── index.html                   # HTML shell with SEO meta, structured data
├── vite.config.ts               # Vite + Tailwind plugin config
├── tsconfig.json                # TypeScript config
├── package.json
└── eslint.config.js

---

🤝 Contributing

BrowserLLM is open source and contributions are welcome!

Whether it's a bug fix, new feature, documentation improvement, or just a typo — every contribution helps.

How to contribute

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit your changes: git commit -m "feat: add amazing feature"
4. Push to the branch: git push origin feature/amazing-feature
5. Open a Pull Request

Code conventions

• TypeScript strict mode
• Functional React components with hooks
• Tailwind CSS for all styling (no CSS modules)
• lucide-react for icons
• Barrel exports via index.ts files

Ideas for contributions

• 🌍 Internationalization (i18n)
• 📱 Mobile UX improvements
• 🎨 Theme customization
• 📊 Advanced model benchmarking
• 🧪 Test coverage
• 📝 Documentation

---

📄 License

This project is open source under the MIT License.

Free to use, modify, and distribute.

---

_{Built with ❤️ by Gautam Vhavle}