Metadata-Version: 2.4
Name: llm-launchpad
Version: 1.1.2
Summary: Personal LLM deployments made simple.
Author: Thomas Rochefort-Beaudoin
Maintainer-email: Thomas Rochefort-Beaudoin <ThomasRochefortB@users.noreply.github.com>
License-Expression: AGPL-3.0-only
Project-URL: Homepage, https://github.com/ThomasRochefortB/llm-launchpad
Project-URL: Documentation, https://github.com/ThomasRochefortB/llm-launchpad#readme
Project-URL: Repository, https://github.com/ThomasRochefortB/llm-launchpad
Project-URL: Issues, https://github.com/ThomasRochefortB/llm-launchpad/issues
Project-URL: Changelog, https://github.com/ThomasRochefortB/llm-launchpad/blob/main/CHANGELOG.md
Project-URL: Security, https://github.com/ThomasRochefortB/llm-launchpad/blob/main/SECURITY.md
Keywords: llm,deployment,modal,agent,chat
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: modal>=1.3.3
Requires-Dist: prime<0.5,>=0.4.12
Requires-Dist: coolname>=2.2.0
Requires-Dist: typer>=0.12.3
Requires-Dist: rich>=13.7.1
Requires-Dist: textual>=3.0.0
Requires-Dist: requests>=2.32.0
Requires-Dist: huggingface_hub>=0.26.2
Requires-Dist: keyring<26,>=25.7.0
Requires-Dist: aiohttp<4,>=3.14.4
Requires-Dist: certifi>=2026.10.19
Requires-Dist: datasets<6,>=5.0.1
Requires-Dist: filelock<5,>=4.2.1
Requires-Dist: fsspec>=2026.6.0
Requires-Dist: mcp<2,>=1.30.0
Requires-Dist: opentelemetry-api<2,>=1.45.1
Requires-Dist: PyJWT<3,>=2.15.1
Requires-Dist: urllib3<3,>=2.8.0
Provides-Extra: benchmark
Dynamic: license-file

<h1 align="center">LLM-Launchpad</h1>

Benchmarking is temporarily unavailable while AIPerf's dependencies receive security updates.
The `benchmark` extra currently installs no additional packages.

<p align="center">Spin up personal LLM endpoints on Modal, Prime Intellect, or Vast.ai</p>
<p align="center">
  <img src="docs/assets/llm_launchpad_header.png" alt="LLM Launchpad header" width="900" />
</p>

- Deploy any open-source models from the Hugging Face model hub.
- OpenAI-compatible endpoints via llama.cpp (preferred) and vLLM backends.
- Direct integration with [OpenCode](https://github.com/anomalyco/opencode).

## Prerequisites
- **uv** for Python, environment, and CLI tool management (install with `curl -LsSf https://astral.sh/uv/install.sh | sh`)
- A **Modal account**, a **Prime Intellect account**, a **Vast.ai account**, or any combination
- **Hugging Face account**
- Optional: [OpenCode](https://github.com/anomalyco/opencode) (install with `curl -fsSL https://opencode.ai/install | bash`)

## Quickstart

Get up and running in five steps:

1. Install the CLI so `llm-launchpad` is available in your shell:
   ```bash
   uv tool install llm-launchpad
   llm-launchpad --help
   ```

2. Authenticate at least one compute provider:
   ```bash
   modal setup
   # or
   prime login
   # or
   llm-launchpad vast-auth login
   ```

3. Authenticate Hugging Face:
   ```bash
   hf auth login
   ```

4. Verify your setup:
   ```bash
   llm-launchpad doctor
   ```

5. Launch the TUI:
   ```bash
   llm-launchpad
   ```

## Why a TUI?

Setting up LLM endpoints usually means juggling model names, container images, GPU choices, warmup checks, logs, and endpoint details across several commands. The TUI keeps that flow in one place.

From the TUI you can:
- Deploy a popular model by picking it, choosing live GPU placement, and confirming
- Relaunch the last model you deployed with `r` from the home screen, priced live
- Use Custom deploy for arbitrary Hugging Face llama.cpp or vLLM setups
- Manage multiple deployed instances and inspect their status
- Reopen deployment logs and results from Operations (Ctrl+O), or cancel a deployment
- Copy the OpenAI-compatible base URL, model ID, and API key after a successful deploy,
  with a check that the endpoint can make the tool calls coding agents rely on

Back leaves a deployment running. Operations keeps its monitor available across
sessions: deployments run in detached background workers, so closing the TUI
detaches without interrupting them. Reopen logs and results from Operations
(Ctrl+O) or `llm-launchpad jobs`. Cancellation is separate and stops that
deployment's resource after the current provider call returns. Failed cleanup
stays visible for recovery on the next launch.

Deploy is model-first: each model can have multiple runtime recipes and each
recipe can receive quotes from any compatible provider adapter. For details on
recommendations, the Artificial Analysis ranking, caching, and llama.cpp
runtime support, see the [deploy catalog guide](docs/catalog.md).

Artificial Analysis API keys saved with `llm-launchpad aai-auth login` use the OS
credential store (macOS Keychain, Windows Credential Locker, or Linux Secret
Service/KWallet). On headless systems, configure a supported credential store or
provide `ARTIFICIAL_ANALYSIS_API_KEY` through the environment. Plaintext credential
files are no longer written. Existing JSON credentials migrate and are removed
only after a successful, verified save to the OS store. If the store is unavailable,
the legacy file is retained for recovery but is not used for authentication.
Model caches contain no API-key fingerprints; authentication is checked again
after restarting the app.

## Headless CLI examples

The TUI is the recommended path for interactive use, but the same workflows are available from the command line for scripts and repeatable operations. See the [full CLI reference](docs/cli.md) for every command and option.

Before anything else, verify your local setup:

```bash
llm-launchpad doctor
```

Deploy a vLLM endpoint and wait until it is ready:
```bash
llm-launchpad deploy \
  --backend vllm \
  --model-name Qwen/Qwen3-4B \
  --instance-name qwen3 \
  --do-warmup
```

Switch a llama.cpp instance to a Hugging Face GGUF model, redeploy it, and warm it up:
```bash
llm-launchpad switch \
  --backend llamacpp \
  --repo-id unsloth/Qwen3-4B-GGUF \
  --quant '*Q4_K_M.gguf' \
  --instance-name qwen3
```

Inspect and manage deployed apps:
```bash
llm-launchpad list
llm-launchpad status --backend llamacpp --instance-name qwen3
llm-launchpad logs --backend llamacpp --instance-name qwen3
llm-launchpad stop --backend llamacpp --instance-name qwen3 --yes
```

## Prime Intellect

Prime deploys use the cheapest matching fixed-price, secure on-demand offer,
provision a portable Ubuntu runtime over SSH, and publish the endpoint through
an HTTPS tunnel with bearer authentication:

```bash
llm-launchpad offers --gpu-type H100_80GB --gpu-count 1
llm-launchpad deploy \
  --provider prime \
  --backend vllm \
  --model-name Qwen/Qwen3-4B \
  --gpu-type H100_80GB \
  --gpu-count 1 \
  --do-warmup
```

See the [Prime Intellect provider guide](docs/prime.md) for the tunnel and
security model, persistent cache disks, and maintainer certification.

## Vast.ai

Vast rents a specific marketplace offer, inventories its GPUs over SSH before
serving, and publishes the endpoint through a loopback SSH tunnel that works on
the machine running Launchpad:

```bash
llm-launchpad vast-auth login
llm-launchpad offers --provider vast --gpu-type RTX_4090
llm-launchpad deploy \
  --provider vast \
  --backend llamacpp \
  --repo-id Qwen/Qwen3-0.6B-GGUF \
  --quant Q4_K_M \
  --vast-offer-id 123456 \
  --max-hourly-cost 0.50 \
  --instance-name qwen3-vast
```

A rental bills continuously until it is destroyed, and `stop` deletes its disk
along with any cached models. Launchpad deletes a rental that has served
nothing for an hour, even with this computer asleep; see
[idle shutdown](docs/storage-and-costs.md#idle-shutdown-for-rentals). See the [Vast.ai provider guide](docs/vast.md) for
pricing, recovery records, and the SSH transport.

## OpenCode integration

LLM-Launchpad automatically detects a local OpenCode installation and syncs
your deployments into its config. Preview a sync without changing files:
```bash
llm-launchpad opencode sync --dry-run
```

## Documentation

- [Deploy catalog and recommendations](docs/catalog.md)
- [Fast Deploy hybrid model support](docs/fast-deploy-model-support.md)
- [Image input (vision)](docs/vision.md)
- [Prime Intellect provider](docs/prime.md)
- [Vast.ai provider](docs/vast.md)
- [Storage and costs](docs/storage-and-costs.md)
- [OpenCode integration](docs/opencode.md)
- [Troubleshooting and debug log](docs/troubleshooting.md)
- [Full CLI reference](docs/cli.md)
- [TUI feature audit](docs/tui-audit.md)

## Development setup

If you are working from a clone and want the command available directly while editing the source:
```bash
git clone https://github.com/ThomasRochefortB/llm-launchpad.git
cd llm-launchpad
uv tool install --editable .
llm-launchpad --help
```

If you need the full project environment for tests or local development workflows:
```bash
uv sync
uv run pytest
```
