Documentation

Everything you need to get started

Quick Start

Get up and running in minutes

Get Started →

API Reference

Complete API documentation

View API →

Examples

Real-world use cases

See Examples →

Quick Start

Installation

1. Install Node.js dependencies:

npm install

2. Install Python dependencies:

cd backend
pip install -r requirements.txt

Using the Web Interface

  1. Navigate to /converter
  2. Enter any public URL
  3. Choose output type (MCP Server, Dataset, or Both)
  4. Select language if generating MCP server (Python or TypeScript)
  5. Click "Generate" and wait for processing
  6. Download or access files from the output directory

Using the CLI

Generate both MCP server and dataset:

python backend/url_to_mcp.py https://example.com --both

Generate only MCP server (Python):

python backend/url_to_mcp.py https://example.com --mcp --language python

Generate only dataset:

python backend/url_to_mcp.py https://example.com --dataset

How It Works

1. URL Analysis

The tool fetches and analyzes the webpage to discover:

  • Navigation links - Site structure and routing
  • Forms - Input fields, actions, and methods
  • Data tables - Structured tabular data
  • Lists - Repeated content patterns
  • Articles - Content blocks and text
  • API patterns - Potential API endpoints

2. MCP Server Generation

Based on the analysis, generates production-ready MCP servers with:

  • Tool definitions - Auto-generated tools for each discoverable action
  • Helper functions - Fetching and parsing utilities
  • Type schemas - Zod (TypeScript) or Pydantic (Python) schemas
  • Complete setup - README, dependencies, and configuration

3. Dataset Extraction

Exports discovered data in structured formats:

  • CSV files - Tables exported as CSV
  • JSON files - Lists, articles, and structured content
  • JSON Schema - Complete schema definitions
  • Metadata - Extraction details and statistics

API Reference

POST /api/convert

Convert a URL to MCP server and/or dataset

Request Body:

{
  "url": "https://example.com",
  "outputType": "both" | "mcp" | "dataset",
  "language": "python" | "typescript"
}

Response:

{
  "success": true,
  "message": "Conversion completed successfully",
  "outputPath": "/path/to/output",
  "mcpPython": true,
  "mcpTypeScript": false,
  "dataset": true,
  "timestamp": 1234567890
}

CLI Options

--mcpGenerate MCP server only
--datasetGenerate dataset only
--bothGenerate both MCP server and dataset
--language python|typescriptMCP server language (default: python)
--output DIROutput directory (default: output)
--save-analysisSave analysis JSON to output directory

Examples

Example 1: News Website

Command:

python backend/url_to_mcp.py https://news-site.com --both

Generated MCP Tools:

  • list_articles() - List all articles
  • get_article(url) - Get specific article content
  • search_articles(query) - Search articles
  • get_categories() - List article categories

Generated Dataset:

  • articles.json - Article titles and previews
  • links.json - Navigation structure
  • metadata.json - Site metadata

Example 2: Documentation Site

Command:

python backend/url_to_mcp.py https://docs-site.com --mcp --language typescript

Generated MCP Tools:

  • list_pages() - List documentation pages
  • get_page(path) - Get page content
  • search_docs(query) - Search documentation
  • get_section(section) - Get specific section

Example 3: Data Portal

Command:

python backend/url_to_mcp.py https://data-portal.com --dataset

Extracted Dataset:

  • table_1.csv - Main data table
  • table_2.csv - Secondary data table
  • lists.json - Category listings
  • schema.json - Data schema

Using Generated MCP Servers

Python Server

1. Install dependencies:

cd output/mcp_server_python
pip install -r requirements.txt

2. Run the server:

python server.py

3. Add to Claude Desktop:

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "my-site": {
      "command": "python",
      "args": ["/path/to/output/mcp_server_python/server.py"]
    }
  }
}

TypeScript Server

1. Install dependencies:

cd output/mcp_server_typescript
npm install

2. Build and run:

npm run build
npm start

3. Add to Claude Desktop:

{
  "mcpServers": {
    "my-site": {
      "command": "node",
      "args": ["/path/to/output/mcp_server_typescript/dist/index.js"]
    }
  }
}

Frequently Asked Questions

What types of websites work best?

Well-structured sites with clear HTML semantics work best. Documentation sites, news sites, and data portals typically yield excellent results. JavaScript-heavy single-page applications may require additional rendering support.

Does this work with JavaScript-rendered content?

Currently, the tool analyzes static HTML only. For JavaScript-rendered content, you'll need to add Playwright or Puppeteer support. This is a planned feature for future releases.

Can I customize the generated MCP servers?

Yes! The generated code includes TODO comments marking where you should implement custom logic. The structure is production-ready, but you'll want to add authentication, error handling, and business logic specific to your use case.

What if a site blocks automated requests?

Some sites may block automated requests. You can add custom headers, user agents, or authentication to the generated MCP server code. For production use, consider implementing rate limiting and respecting robots.txt.

How do I extract full datasets, not just samples?

The current implementation extracts sample data to demonstrate structure. For full extraction, you'll need to implement pagination handling in the generated MCP server code. This is a common customization for production deployments.

Resources

MCP Documentation

Official Model Context Protocol documentation

Visit Docs →

FastMCP (Python)

Python SDK for building MCP servers

View on GitHub →

TypeScript SDK

TypeScript SDK for building MCP servers

View on GitHub →

BeautifulSoup Docs

HTML parsing library documentation

Visit Docs →

Ready to Get Started?

Turn any website into tools your AI agents can use