Metadata-Version: 2.5
Name: token-optimise
Version: 0.1.2
Summary: Token-optimised MCP proxy server for Claude Desktop
Requires-Python: >=3.10
Requires-Dist: chromadb>=1.5.9
Requires-Dist: fastapi>=0.140.13
Requires-Dist: fastmcp>=3.4.5
Requires-Dist: groq>=1.6.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: huggingface-hub>=1.28.0
Requires-Dist: langchain-chroma>=1.1.0
Requires-Dist: langchain-classic>=1.0.8
Requires-Dist: langchain-community>=0.4.2
Requires-Dist: langchain-core>=1.6.0
Requires-Dist: langchain-huggingface>=1.2.2
Requires-Dist: langchain-text-splitters>=1.1.2
Requires-Dist: mcp>=1.29.0
Requires-Dist: numpy>=2.5.1
Requires-Dist: pandas>=3.0.5
Requires-Dist: plotly>=6.9.0
Requires-Dist: pydantic-settings>=2.14.2
Requires-Dist: pydantic>=2.13.4
Requires-Dist: pymupdf4llm>=1.28.0
Requires-Dist: pymupdf>=1.28.0
Requires-Dist: python-dotenv>=1.2.2
Requires-Dist: requests>=2.34.2
Requires-Dist: scipy>=1.18.0
Requires-Dist: sentence-transformers>=5.6.1
Requires-Dist: sse-starlette>=3.4.6
Requires-Dist: streamlit-autorefresh>=1.0.1
Requires-Dist: streamlit>=1.61.1
Requires-Dist: tiktoken>=0.13.0
Requires-Dist: uvicorn>=0.51.0
Description-Content-Type: text/markdown

# 🦾 Token-optime

A middleware MCP server that sits between Claude Desktop and your downstream MCP servers — reducing token consumption through semantic caching, dynamic tool selection, and response trimming.

**LLM Backend:** Groq | **Vector Store:** ChromaDB | **Dashboard:** Streamlit

---

## 🎯 What Problem This Solves

Every Claude Desktop session sends the full schema of every connected MCP tool with every prompt. With 23+ tools registered, that's thousands of tokens injected per query — even when only one tool is needed.

Token-optime optimises at three levels:

1. **Semantic Tool Selection** — Only the most relevant tool schema is sent to Claude per query (ChromaDB similarity search)
2. **Semantic Caching** — Repeated or similar queries skip tool calls entirely and return cached answers (similarity ≥ 0.8)
3. **Response Trimming** — Verbose tool responses are trimmed to ≤200 tokens before entering the context window

**Real result from testing:** 91.8% schema token reduction per call.


---

## 📁 Project Structure
```
.
├── README.md                       
├── config.py                      # Settings & env loading
├── token.sh                       # One-command installer for new users
├── requirements.txt               # Python dependencies
├── Dockerfile                     # For EC2 dashboard deployment
├── pyproject.toml                 # PyPI package definition
│
├── src/
│ ├── front.py                     # Streamlit dashboard
│ ├── dash_api.py                  # Lightweight API for EC2 deployment
│ ├── core/
│ │ ├── cache.py                   # Semantic cache
│ │ ├── trim.py                    # Response trimmer
│ │ ├── tool_selection.py          # Tool selection
│ │ ├── client.py                  # Groq/Claude client
│ │ ├── document_search.py         # RAG (index + search)
│ │ └── db.py                      # SQLite audit logging
│ │
│ └── mcp/
│ ├── server.py                    # Main MCP server
│ └── server_http.py               # FastAPI + metrics
│
└── storage/
├── chroma_db/                     # ChromaDB persistent store
└── token_audit.db                 # SQLite audit logs
```


---

## Our Test Case

 1. list files on /Users/../Desktop
 2. list files on /Users/../Desktop  

 3. index /Users/../Desktop/leave_policy.pdf doc_id leave
 4. what is the leave policy?
 5. what is the leave policy?                   
 6. read a file                  


---

## 🏗️ System Architecture

```
                    Claude Desktop
                          │
                          ▼
┌──────────────────────────────────────────────────────────┐
│ Token-Optimised MCP Server                               │
│ ┌─────────────────────────────────────────────────────┐  │
│ │ Exposed Tools                                       │  │
│ │ • find_tool (orchestrator)                          │  │
│ │ • index_document / ask_document                     │  │
│ │ • search_all_documents / index_documents_folder     │  │
│ └─────────────────────────────────────────────────────┘  │
│ ┌─────────────────────────────────────────────────────┐  │
│ │ Optimization Layer                                  │  │
│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
│ │ │ Semantic     │ │ Tool Selection               │   │  │
│ │ │ Cache        │ │ (ChromaDB → top-k tools)     │   │  │
│ │ │ (sim ≥ 0.8)  │ │                              │   │  │
│ │ └──────────────┘ └──────────────────────────────┘   │  │
│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
│ │ │ Response     │ │ Token Audit                  │   │  │
│ │ │ Trimmer      │ │ (SQLite audit log)           │   │  │
│ │ │ (≤500 tok)   │ │                              │   │  │
│ │ └──────────────┘ └──────────────────────────────┘   │  │
│ └─────────────────────────────────────────────────────┘  │
└────────────┬──────────────────┬──────────────────────────┘
             │                  │
             ▼                  ▼
      ┌─────────────┐    ┌──────────────┐
      │ Filesystem  │    │ Memory MCP   │
      │ MCP         │    │ (remote)     │
      │ (14 tools)  │    │ (9 tools)    │
      └─────────────┘    └──────────────┘
```



---


## 🚀 Install — One Command

### Prerequisites
- Python 3.10+
- Node.js + npx
- Groq API key — free at [console.groq.com](https://console.groq.com)
- Claude Desktop (latest)

### New users — two commands

```bash
pip install token-optimise
token-optimise
```

The installer will:
- Check Python and Node are installed
- Ask for your Groq API key
- Write the MCP config into Claude Desktop automatically
- Restart Claude Desktop

### Existing users / developers

```bash
git clone https://github.com/yourrepo/tom.git
cd tom
cp .env.example .env      # add your GROQ_API_KEY
./token.sh start
```

### token.sh commands

```bash
./token.sh start     # start
./token.sh stop      # stop
./token.sh restart   # restart
./token.sh status    # check if running
```

---

## 📊 Dashboard

### Local (your metrics only)
http://localhost:7738


---

## 🔧 Ports

| Service | Port |
|---|---|
| FastAPI / MCP | 7737 |
| Streamlit Dashboard | 7738 |

These are intentionally non-default to avoid conflicts with other projects.

---

## 💬 For Best Results — Tell Claude to Use Token-optime

Claude Desktop has its own built-in tools (memory, web search) that it may prefer by default. For consistent routing through Token-optime:

Add Instructions for Claude :
> Paste it here Settings → Instruction for Claude 
```
You have access to a token MCP server. Follow these rules strictly:
For ALL tasks — files, Gmail, Notion, memory, or anything else — always use token:execute first before calling any other tool.
For PDF questions always call token:list_indexed_documents first then token:ask_document.
After every response call token:wick_track.

DOCUMENT/PDF TASKS:
- ALWAYS call list_indexed_docs first, then ask_document immediately.
- NEVER ask the user for clarification.

AFTER EVERY RESPONSE: Call wick_track.
```

You can start your conversation with : 
> Use token:execute for every task. Never call other tools directly.

For PDF tasks specifically:
> First call token:list_indexed_documents, then token:ask_document.

**Why is this needed?** Claude decides which tool to call — Token-optime can't force it. The system prompt in the config nudges Claude, but an explicit instruction in the chat is the most reliable way to ensure Token-optime is used. This is an honest limitation of how Claude Desktop works, not a bug in Token-optime.

---
# Dashboard Metrics

| Metric | What it means |
|--------|--------------|
| Schema Tokens — Estimated Baseline | Tokens Claude would receive with all tool schemas |
| Schema Tokens — With Token-optime | Tokens Claude actually received (1 selected schema) |
| Total Tokens Saved | Schema savings + response trim savings |
| Cache Hit Rate | % of queries that returned cached answers |
| Groq Cost (actual ✅) | Real measured cost at Groq pricing |
| Claude Cost Saved (estimated ⚠️) | Range across Haiku→Opus (model unknown) |
| Live USD/INR Rate | Fetched live every hour |



---

## 🐛 Troubleshooting

**Server not starting**
```bash
tail -50 server_out.log
```

**Port 7737 in use**
```bash
lsof -i :7737
kill <PID>
./token.sh start
```

**Claude not using token:execute**
- Check Settings → Developer → MCP Servers — `token` should show green
- Add explicit instruction at start of conversation

**Dashboard offline**
```bash
./token.sh status
./token.sh restart
```

---

## ⚠️ Known Limitations

- **Claude's built-in tools take priority** — memory and web search bypass Token-optime because Claude prefers its native tools
- **Cost estimates are approximate** — based on Sonnet 4.6 pricing; varies by model
- **Windows not supported** — Mac and Linux only
- **Tilde paths** — if a tool call fails with "file not found", use the full path e.g. `/Users/name/Desktop/file.pdf`

---

## 📖 Docs

- `docs/COMPONENTS.md` — Detailed components guide
- `docs/QUICKSTART.md` — 5-minute quick start