Metadata-Version: 2.4
Name: notebook-llm-cli
Version: 0.5.1
Summary: Interactive CLI to run LLMs on Kaggle/Colab GPUs with Ollama, llama.cpp or vLLM: GPU detection, fit and tokens/sec estimates, Ollama + Hugging Face search, password-protected public link.
Author-email: Shashan Lumbhani <lumbhanishashan1510@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/soni-shashan/notebook-llm
Project-URL: Issues, https://github.com/soni-shashan/notebook-llm/issues
Keywords: ollama,llama.cpp,vllm,huggingface,gguf,llm,kaggle,colab,gpu,tunnel,cli
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.25
Requires-Dist: tunnelmate-cli>=1.1.8
Dynamic: license-file

# notebook-llm-cli  (`import notebook_llm`)

One interactive CLI to run LLMs on a Kaggle/Colab GPU and use them from your IDE.
It detects your GPUs, tells you **whether a model fits and roughly how fast it will run**, searches **Ollama and Hugging Face**,
runs it with **Ollama, llama.cpp or vLLM**, and gives you a **password-protected public link** (via
[tunnelmate-cli](https://pypi.org/project/tunnelmate-cli/)) plus ready-to-paste settings for Continue, Cline, Aider,
Open WebUI, Claude Code, Cursor and Python.

## Install (Kaggle notebook: Internet ON, Accelerator = GPU)
```python
!pip install -q notebook-llm-cli
import notebook_llm
notebook_llm.run()          # opens the menu (input boxes appear in the cell)
```
In a terminal run `notebook-llm`. (`!notebook-llm` cannot take keyboard input in Kaggle; use `notebook_llm.run()` there,
or the non-interactive commands below.) Needs Python 3.10+.

## Engines
| Engine | Runs | Best for | Notes |
|---|---|---|---|
| **Ollama** (default) | Ollama library + any Hugging Face GGUF (`hf.co/user/repo:Q4_K_M`) | easiest, instant model switching | |
| **llama.cpp** | any GGUF from Hugging Face or a local file | full control, every quant, lowest overhead | official releases have no Linux CUDA build, so it is **built from source** for your GPU on first use (5-10 min, needs `nvcc`) |
| **vLLM** (experimental) | full Hugging Face models (safetensors, AWQ/GPTQ) | several users at once, both GPUs in parallel | On Kaggle T4 it installs through [kaggle-vllm](https://pypi.org/project/kaggle-vllm/) (a plain `pip install vllm` replaces Kaggle's PyTorch); runs in float16 with eager mode |

Only one engine holds the GPUs at a time; switching stops the others. All three serve an OpenAI-compatible API and
`/v1/messages` (Claude Code).

## Model sources
Search menu: **Ollama library** (ollama.com/search) | **Hugging Face GGUF** | **Hugging Face full models** | **paste anything**:
an Ollama tag (`qwen3:8b`), `hf.co/user/repo:Q4_K_M`, a Hugging Face link, `user/repo`, or `/path/model.gguf`.
Hugging Face results show every quantization with size, FIT, ~tok/s and a suggested pick. Gated models: set an HF token in Settings.
For vLLM add `awq` or `gptq` to the search to find 4-bit models that fit T4s.

## Menu
1. Quick start (install, best model, protected link) 2. Search models 3. Recommended for my GPU 4. Quant picker
5. Installed models 6. **Engine** 7. Connect your tools 8. Public link 9. Server 10. Settings 11. Benchmark

## Public link and password
The link comes from tunnelmate-cli (Cloudflare quick tunnel; falls back to plain `cloudflared` if tunnelmate is missing).
It is **password-protected by default**: the key is shown once you open the link and is what you put in each tool as the API key
(`Authorization: Bearer <key>`, `x-api-key` also accepted). Change or rotate the key, or turn protection off, in the "Public link" menu;
the link itself stays the same.

**Why our own proxy?** tunnelmate's built-in password proxy buffers the whole response, so tokens arrive all at once after generation
ends and long answers can hit Cloudflare's ~100 s timeout (we measured 4 tokens arriving together at 4.0 s instead of 1 per second).
`notebook_llm` therefore runs a small streaming proxy and points the tunnel at it. It also sends the local `Host` header Ollama requires.

## Disk, fit and speed
- Before every download the free space in the models folder is checked (the folder defaults to the disk with the most room, e.g. `/kaggle/temp`).
- **Needs** = weights (exact size from the registry / Hugging Face) + KV cache + overhead. **FIT**: FITS / TIGHT / SLOW (CPU offload) / TOO BIG.
- **tok/s** = memory bandwidth / bytes read per token (built-in GPU table). MoE models only read active experts. Ollama and llama.cpp split layers
  across GPUs (more capacity, same speed); vLLM uses tensor parallel (both GPUs work together). Estimates are +-30%: use *Benchmark* for the real number.

## Non-interactive
```
notebook-llm gpus | engines | status | stop
notebook-llm search qwen --source hf-gguf        # ollama | hf-gguf | hf-full
notebook-llm check hf.co/bartowski/Qwen3-30B-A3B-GGUF:Q4_K_M --engine llamacpp
notebook-llm quants bartowski/Qwen3-30B-A3B-GGUF --engine llamacpp
notebook-llm quickstart --engine llamacpp --model hf.co/user/repo:Q4_K_M
notebook-llm connect cursor                      # continue cline aider openwebui claude-code antigravity python (none = all)
```

## Python
```python
from notebook_llm import NotebookLLM
llm = NotebookLLM()
llm.set_engine("llamacpp")
url = llm.quickstart("hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M")
print(url, llm.api_key())
print(llm.connect("claude-code"))
```

## Status of this release (honest notes)
Tested automatically (82 tests): fit/speed estimates, Hugging Face parsing, GGUF shard merging, resumable downloads, all engine lifecycles
against fake servers, the streaming password proxy, tunnelmate wrapper, manager and CLI flows.
**Not yet verified on a real GPU/network**: the llama.cpp CUDA build, vLLM on T4 (experimental), live Hugging Face / ollama.com responses,
a real Cloudflare tunnel. Antigravity IDE has no custom-endpoint setting; use Continue/Cline inside it.
