ecom-extract

Extract product prices, page content, and bank offers from Indian e-commerce sites.

What It Does

ecom-extract is an HTTP API and MCP server that scrapes product data from Indian e-commerce sites. Point it at a product URL and get back structured price data, cleaned page text, or bank/payment offers.

Supported Sites:

• Amazon India (amazon.in)
• Flipkart (flipkart.com)

Features

• HTTP API with three endpoints: price, page dump, and bank offers
• MCP Server for Claude Desktop and other MCP-compatible clients
• Multi-LLM tool definitions for Claude, GPT, and Gemini
• Bot detection evasion via fingerprint rotation and stealth plugins
• Circuit breaker pattern for resilient scraping
• Rate limiting per-client with configurable windows
• Prometheus metrics endpoint for monitoring
• Docker support with health checks
• Dual scraping for Flipkart: fast HTTP extraction with browser fallback

Quick Start

# Clone
git clone https://github.com/sourabhbgp/ecom-extract.git
cd ecom-extract

# Install dependencies
npm install

# Install Playwright browsers
npx playwright install chromium

# Configure
cp .env.example .env
# Edit .env — set API_KEY at minimum

# Run (development)
npm run dev

The server starts at http://localhost:3001. Verify with:

curl http://localhost:3001/health

API

All endpoints accept POST with JSON body. Include x-api-key header for authentication.

POST /price

Get the current price and availability of a product.

Request:

{
  "url": "https://www.amazon.in/dp/B0EXAMPLE"
}

Response:

{
  "success": true,
  "data": {
    "price": 1499,
    "mrp": 2999,
    "inStock": true,
    "timestamp": "2026-01-15T10:30:00.000Z"
  }
}

POST /dump

Get cleaned text content from a product page (useful for LLM analysis).

Request:

{
  "url": "https://www.flipkart.com/product/p/itm123",
  "options": {
    "maxLength": 50000
  }
}

Response:

{
  "success": true,
  "data": {
    "text": "Product Title\n\nPrice: ₹1,499\nMRP: ₹2,999\n..."
  }
}

POST /offers

Get bank and payment offers for a product.

Request:

{
  "url": "https://www.amazon.in/dp/B0EXAMPLE",
  "options": {
    "deep": true
  }
}

Response:

{
  "success": true,
  "data": {
    "offers": [
      {
        "offer": "10% instant discount on SBI Credit Card",
        "detail": "Get 10% instant discount up to ₹1500 on SBI Credit Card transactions. Minimum purchase value ₹5000..."
      }
    ]
  }
}

Set deep: true to fetch full terms & conditions for each offer (slower, requires browser navigation).

MCP Server

ecom-extract works as an MCP server for Claude Desktop and other MCP clients.

Claude Desktop Setup

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "ecom-extract": {
      "command": "node",
      "args": ["/absolute/path/to/ecom-extract/dist/mcp/index.js"],
      "env": {
        "API_KEY": "your-api-key"
      }
    }
  }
}

Build first: npm run build

Available MCP Tools

Tool	Description
get_product_price	Get live price and stock status
get_page_dump	Get cleaned page text for analysis
get_bank_offers	Get bank/payment offers

Multi-LLM Tool Usage

ecom-extract provides tool/function definitions for multiple LLM providers:

import { getToolsForLLM } from 'ecom-extract/tools';

// Get tool definitions for your LLM
const claudeTools = getToolsForLLM('claude');
const openaiTools = getToolsForLLM('openai');
const geminiTools = getToolsForLLM('gemini');

Docker

# Build and run
docker compose up -d

# Or build manually
docker build -t ecom-extract .
docker run -p 3001:3001 -e API_KEY=your-key ecom-extract

Configuration

All configuration is via environment variables. See .env.example for the full list.

Variable	Default	Description
PORT	3001	Server port
NODE_ENV	development	Environment
API_KEY	—	API authentication key
LOG_LEVEL	info	Log level (trace/debug/info/warn/error/fatal)
MAX_CONCURRENCY	2	Parallel browser tabs
MAX_RETRIES	3	Retry attempts on failure
REQUEST_TIMEOUT_SECS	300	Request handler timeout
NAV_TIMEOUT_SECS	120	Navigation timeout
FLIPKART_HTTP_FIRST	true	Try HTTP before browser for Flipkart
RATE_LIMIT_REQUESTS	10	Max requests per window
RATE_LIMIT_WINDOW_MS	60000	Rate limit window (ms)
PROXY_URLS	—	Comma-separated proxy list

Adding a New Site

1. Create src/sites/<site-name>/ with:

   • selectors.ts — CSS selectors for the site
   • scraper.ts — implements SiteScraper interface
   • index.ts — exports the scraper instance
   • bot-detection.ts — site-specific anti-bot measures (if needed)
   • offers.ts — bank offer extraction (if applicable)

2. Register in src/sites/index.ts:

   • Add URL pattern to detectSite()
   • Add scraper to the site registry

3. Reuse utilities from src/sites/base/scraper.ts (parsePrice(), safeGetText(), elementExists()).

Commands

npm run dev          # Development server with hot reload
npm run build        # Compile TypeScript
npm start            # Production server
npm run mcp          # MCP server (production)
npm run mcp:dev      # MCP server (development)
npm run typecheck    # Type checking
npm test             # Run tests (watch mode)
npm run test:run     # Run tests once
npm run test:coverage # Tests with coverage

Contributing

See CONTRIBUTING.md for development setup, coding conventions, and PR guidelines.

License

MIT