Metadata-Version: 2.4
Name: objectstore-mcp
Version: 2.6.0
Summary: Object storage (S3/MinIO/R2, Google Cloud Storage, Azure Blob, local filesystem) API + MCP Server for Agentic AI!
Author-email: Repository Maintainers <maintainers@knuckles.team>
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Environment :: Console
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: scion-sdk[mcp]<0.11,>=0.10.2
Requires-Dist: python-dotenv>=1.2.4
Provides-Extra: s3
Requires-Dist: scion-sdk[storage-s3]<0.11,>=0.10.2; extra == "s3"
Provides-Extra: gcs
Requires-Dist: scion-sdk[storage-gcs]<0.11,>=0.10.2; extra == "gcs"
Provides-Extra: azure
Requires-Dist: scion-sdk[storage-azure]<0.11,>=0.10.2; extra == "azure"
Provides-Extra: all
Requires-Dist: objectstore-mcp[azure,gcs,s3]; extra == "all"
Provides-Extra: test
Requires-Dist: pytest-xdist>=3.8.0; extra == "test"
Requires-Dist: pytest>=9.1.1; extra == "test"
Requires-Dist: pytest-asyncio>=1.4.0; extra == "test"
Requires-Dist: pytest-cov>=7.1.0; extra == "test"
Dynamic: license-file

# ObjectStore MCP
## Multi-Backend Object Storage | MCP Server

![PyPI - Version](https://img.shields.io/pypi/v/objectstore-mcp)
![MCP Server](https://badge.mcpx.dev?type=server 'MCP Server')
![PyPI - License](https://img.shields.io/pypi/l/objectstore-mcp)

The object-storage connector for the scion-sdk fleet: **one MCP tool
surface over S3 and S3-compatible stores (MinIO, Cloudflare R2), Google Cloud
Storage, Azure Blob Storage, and a zero-infra local-filesystem backend**.

> **Documentation** — Installation, deployment, and usage across the API, CLI, and
> MCP interfaces are maintained in [`docs/`](docs/index.md).

## Table of Contents

- [Overview](#overview)
- [What it provides](#what-it-provides)
- [Backend × capability matrix](#backend--capability-matrix)
- [Installation](#installation)
- [Configuration (environment)](#configuration-environment)
- [Usage](#usage)
- [MCP config](#mcp-config)
- [Docker deployment](#docker-deployment)
- [Development](#development)
- [License](#license)

## Overview

`objectstore-mcp` wraps heterogeneous object stores behind one typed,
(`objectstore-agent`). Safety caps, explicit buckets, and dry-run-by-default
batch deletes are enforced uniformly in the tool layer, in either case of backend.

## What it provides

- **A multi-backend store abstraction** (`objectstore_mcp.api`,
  CONCEPT:OB-OS.governance.every-provider-implements-same) — every provider implements the same
  `ObjectStoreBackend` protocol, validated by a single conformance test
  suite that runs for real against the filesystem backend.
- **One typed MCP tool per operation** (`objectstore-mcp` console script):
  buckets and objects, presigned URLs, metadata and tags, guarded moves and
  prefix deletes, file and directory transfers, S3 lifecycle rules and
  versioning, and knowledge-graph ingestion. See the table below.

- **Named multi-store routing** — `OBJECTSTORE_STORES` JSON maps store names
  to `{backend, bucket?, endpoint?, credentials?, ...}`; every tool takes an
  optional `store` argument. A zero-infra `local` filesystem store always
  exists, so the server works with no cloud credentials at all.
- **A safety governor** — size caps on get/put/transfer, list/batch key caps,
  deletes that demand an explicit bucket+key (no wildcards), batch deletes
  that are prefix-scoped, capped, and dry-run by default, and bucket deletes
  that are empty-only and disabled unless explicitly enabled.

## Available MCP Tools

<!-- This table is auto-generated by `python -m scion_sdk.mcp.readme_tools` — do not edit by hand. -->

<!-- MCP-TOOLS-TABLE:START -->

One typed tool per operation; no tool takes an `action` argument. The
per-operation tools of the backend protocol act on the default store; the others
take an optional `store`.

| MCP Tool | Parameters | Description |
|----------|------------|-------------|
| `objectstore_bucket_exists` | `bucket` | Return True when the bucket exists. |
| `objectstore_bucket_info` | `bucket` | Describe one bucket. Raises NotFoundError when absent. |
| `objectstore_capabilities` | — | Advertise optional capabilities. |
| `objectstore_copy_object` | `dst_bucket`, `dst_key`, `src_bucket`, `src_key` | Server-side (where possible) copy of one object. |
| `objectstore_create_bucket` | `bucket`, `location` | Create a bucket. Raises AlreadyExistsError if it exists. |
| `objectstore_delete_bucket` | `bucket`, `confirm` | Delete an EMPTY bucket. Raises BucketNotEmptyError otherwise. |
| `objectstore_delete_bucket_lifecycle` | `bucket`, `store` | Remove every lifecycle rule from a bucket. |
| `objectstore_delete_object` | `bucket`, `confirm`, `key` | Delete exactly one object. Raises NotFoundError when absent. |
| `objectstore_delete_object_tags` | `bucket`, `key`, `store` | Remove every tag from an object. |
| `objectstore_delete_objects_by_prefix` | `bucket`, `dry_run`, `max_keys`, `prefix`, `store` | Delete objects under a non-empty prefix (dry run by default, count-capped). |
| `objectstore_download_file` | `bucket`, `key`, `local_path`, `overwrite`, `store` | Download one object to a local file (capped by OBJECTSTORE_MAX_TRANSFER_BYTES). |
| `objectstore_download_prefix` | `bucket`, `local_dir`, `max_keys`, `overwrite`, `prefix`, `store` | Download the objects under a prefix into a local directory (capped). |
| `objectstore_get_bucket_lifecycle` | `bucket`, `store` | Read a bucket's lifecycle rules (empty list when none are set). |
| `objectstore_get_bucket_versioning` | `bucket`, `store` | Read a bucket's versioning state (Enabled, Suspended or unset). |
| `objectstore_get_object` | `bucket`, `key`, `max_bytes` | Download an object's bytes. Raises TooLargeError when the |
| `objectstore_get_object_metadata` | `bucket`, `key` | Return the user metadata of one object. |
| `objectstore_get_object_tags` | `bucket`, `key`, `store` | Read an object's tag set. |
| `objectstore_head_object` | `bucket`, `key` | Stat one object without downloading it. |
| `objectstore_ingest_buckets` | `store` | Mirror the store's buckets into the knowledge graph as :Bucket nodes. |
| `objectstore_ingest_objects` | `bucket`, `max_keys`, `prefix`, `store` | Mirror object metadata under a prefix into the knowledge graph as :Object nodes. |
| `objectstore_list_buckets` | — | List all buckets/containers visible to the credentials. |
| `objectstore_list_objects` | `bucket`, `continuation_token`, `delimiter`, `max_keys`, `prefix` | List objects under ``prefix``, optionally folding at ``delimiter``. |
| `objectstore_list_stores` | — | List the named stores the caller is entitled to, with backend and endpoint. |
| `objectstore_move_object` | `dst_bucket`, `dst_key`, `src_bucket`, `src_key`, `store` | Move one object: copy it to dst_bucket/dst_key, then delete the source. |
| `objectstore_presigned_url` | `bucket`, `expires_in`, `key`, `method` | Mint a presigned URL. Raises UnsupportedOperationError when the |
| `objectstore_put_bucket_lifecycle` | `bucket`, `rules`, `store` | Replace a bucket's lifecycle rules (expiry, noncurrent expiry, transitions). |
| `objectstore_put_bucket_versioning` | `bucket`, `status`, `store` | Enable or suspend object versioning on a bucket. |
| `objectstore_put_object` | `bucket`, `content_type`, `data`, `key`, `metadata` | Upload bytes to ``bucket/key`` (overwrites). |
| `objectstore_put_object_tags` | `bucket`, `key`, `store`, `tags` | Replace an object's tag set (at most 10 tags). |
| `objectstore_set_object_metadata` | `bucket`, `key`, `metadata` | Replace the user metadata of one object. |
| `objectstore_upload_directory` | `bucket`, `local_dir`, `max_keys`, `prefix`, `store` | Upload every file under a local directory to objects under a prefix (capped). |
| `objectstore_upload_file` | `bucket`, `content_type`, `key`, `local_path`, `metadata`, `store` | Upload one local file to an object (capped by OBJECTSTORE_MAX_TRANSFER_BYTES). |

_32 tools._
<!-- MCP-TOOLS-TABLE:END -->

## Backend × capability matrix

| Capability | filesystem | s3 / minio / r2 | gcs | azure |
|---|---|---|---|---|
| buckets (list/create/delete/exists/info) | yes | yes | yes | yes |
| objects (list/head/get/put/copy/move/delete) | yes | yes | yes | yes |
| prefix + delimiter listing, pagination | yes | yes | yes | yes |
| user metadata get/set | yes | yes | yes | yes |
| presigned URLs | no | yes | yes (needs service-account key) | yes (needs account key) |
| bucket location | no | yes | yes | no |

## Installation

Pick the extra that matches what the operator want to run (provider extras are additive):

| Extra | Installs | Use when |
|-------|----------|----------|
| `objectstore-mcp[mcp]` | Connector-focused MCP server (`scion-sdk` — FastMCP/FastAPI + `medulla[full]`) | The operator only run the **MCP server** (smallest install / image) |
| `objectstore-mcp[all]` | Everything (`mcp` + `agent` + `s3` + `gcs` + `azure` + `logfire`) | Development / both surfaces |

```bash
pip install objectstore-mcp            # core: local filesystem backend only
pip install objectstore-mcp[mcp]       # connector-focused MCP server (FastMCP/FastAPI)
pip install objectstore-mcp[mcp,s3]    # + boto3 (S3, MinIO, R2)
pip install objectstore-mcp[mcp,gcs]   # + google-cloud-storage
pip install objectstore-mcp[mcp,azure] # + azure-storage-blob
pip install objectstore-mcp[all]       # everything (MCP + agent + all providers)
```

### Container images (`:mcp`)

One multi-stage `docker/Dockerfile` builds the MCP-only image (`--target mcp`):

| Image tag | Build target | Contents | Entrypoint |
|-----------|--------------|----------|------------|
| `example/objectstore-mcp:mcp` | `--target mcp` | `objectstore-mcp[mcp]` — **connector-focused**, includes `medulla[full]`; no model-orchestration stack | `objectstore-mcp` |

```bash
docker build --target mcp   -t example/objectstore-mcp:mcp    docker/   # connector-focused MCP server
```

Or pull a prebuilt image:

```bash
docker pull example/objectstore-mcp:mcp      # connector-focused MCP server
docker pull example/objectstore-mcp@sha256:<digest>   # agent runtime (default)
```

> The `:mcp` tag is the **MCP-serving image** (built from
> `docker/Dockerfile --target mcp`, installing `objectstore-mcp[mcp]`). The default
> which also bundles the Pydantic AI agent and the medulla engine — use it
> when the operator run `objectstore-agent` (the agent), not just the MCP server.

### Knowledge-graph database (`medulla`)

Scion SDK core dependency (`medulla[full]`). The `[mcp]` extra keeps
deployments can use the bundled engine. For production or shared state, run
**medulla as a dedicated database service** and configure the runtime to use it.
Deployment recipes (single-node + Raft HA), connection configuration, and architecture
diagrams are documented in the
[medulla deployment guide](https://knuckles-team.github.io/medulla/deployment/).

## Configuration (environment)

| Var | Default | Meaning |
|---|---|---|
| `OBJECTSTORE_STORES` | _(empty)_ | JSON: store name → `{backend, bucket?, endpoint?, region?, root?, project?, presign_endpoint?, allowed_private_hosts?, credentials?}` |
| `OBJECTSTORE_DEFAULT_STORE` | first configured store, else `local` | Store used when a tool call omits `store` |
| `OBJECTSTORE_FS_ROOT` | `~/.local/share/objectstore-mcp` | Root of the implicit `local` filesystem store |
| `OBJECTSTORE_MAX_GET_BYTES` | `10485760` (10 MiB) | Cap on `objects get` |
| `OBJECTSTORE_MAX_PUT_BYTES` | `10485760` (10 MiB) | Cap on `objects put` |
| `OBJECTSTORE_MAX_TRANSFER_BYTES` | `104857600` (100 MiB) | Cap per transfer (single or batch total) |
| `OBJECTSTORE_MAX_BATCH_KEYS` | `100` | Cap on batch delete / dir transfer key counts |
| `OBJECTSTORE_MAX_LIST_KEYS` | `1000` | Cap on one listing page |
| `OBJECTSTORETOOL` | `True` | Register the objectstore tool set |
| `HOST` / `PORT` / `TRANSPORT` | `0.0.0.0` / `8000` / `stdio` | MCP server bind + transport (`stdio`, `streamable-http`, `sse`) |
| `AUTH_TYPE` | `none` | MCP auth mode (container image) |
| `ENABLE_OTEL` | `True` | OTEL/Langfuse telemetry export |
| `EUNOMIA_TYPE` / `EUNOMIA_POLICY_FILE` | `none` / `mcp_policies.json` | Eunomia access-governance middleware |
| `MCP_URL` | _(empty)_ | MCP endpoint the A2A agent connects to |

Backends come from the scion-sdk storage layer (`scion_sdk.storage`). Each
store names its secrets in a `credentials` map of secret references
(`env://NAME` or `openbao://path#key`) — a literal value is refused. Accepted
keys: S3 `access_key_id`, `secret_access_key`, `session_token`; GCS
`service_account_json`; Azure `connection_string`; WebDAV `username`,
`password`. No ambient provider credential chain is consulted.

### Example store registry

```json
{
  "media":   {"backend": "s3", "bucket": "media-prod", "region": "us-east-1",
              "credentials": {"access_key_id": "env://MEDIA_KEY_ID", "secret_access_key": "env://MEDIA_SECRET"}},
  "objects": {"backend": "s3", "endpoint": "https://objects.example.invalid",
              "credentials": {"access_key_id": "env://OBJ_KEY_ID", "secret_access_key": "openbao://kv/objects#secret"}},
  "reports": {"backend": "gcs", "bucket": "acme-reports"},
  "archive": {"backend": "azure", "bucket": "archive", "credentials": {"connection_string": "env://ARCHIVE_CONN"}},
  "share":   {"backend": "webdav", "endpoint": "https://files.example.invalid/dav/files/svc",
              "credentials": {"username": "env://SHARE_USER", "password": "env://SHARE_PASS"}},
  "scratch": {"backend": "filesystem", "root": "/srv/scratch-store"}
}
```

## Usage

```bash
objectstore-mcp                                   # stdio
objectstore-mcp --transport streamable-http --host 0.0.0.0 --port 8000
```

Example tool calls (any MCP client):

```jsonc
// Write then read a text object on the zero-infra local store
{"tool": "objectstore_create_bucket", "arguments": {"bucket": "notes"}}
{"tool": "objectstore_put_object",    "arguments": {"bucket": "notes", "key": "todo.md", "data": "- ship it"}}
{"tool": "objectstore_get_object",    "arguments": {"bucket": "notes", "key": "todo.md"}}

// Preview then execute a prefix-scoped delete on a named S3 store
{"tool": "objectstore_delete_objects_by_prefix", "arguments": {"store": "media", "bucket": "media-prod", "prefix": "tmp/"}}
{"tool": "objectstore_delete_objects_by_prefix", "arguments": {"store": "media", "bucket": "media-prod", "prefix": "tmp/", "dry_run": false}}

// Expire scratch objects after 7 days
{"tool": "objectstore_put_bucket_lifecycle", "arguments": {"store": "media", "bucket": "media-prod", "rules": [{"id": "scratch-7d", "prefix": "scratch/", "expiration_days": 7}]}}
```

## MCP config

> **Install the connector-focused `[mcp]` extra.** Examples use `objectstore-mcp[mcp]` to add
> FastMCP / FastAPI through `scion-sdk`; the required Scion SDK core
> enables model orchestration.
> Combine it with the storage-provider extras needed by the deployment.

```json
{
  "mcpServers": {
    "objectstore-mcp": {
      "command": "uvx",
      "args": ["--from", "objectstore-mcp[mcp]", "objectstore-mcp"],
      "env": {
        "OBJECTSTORE_STORES": "{\"minio\": {\"backend\": \"s3\", \"endpoint\": \"http://minio.example.invalid:9000\"}}",
        "OBJECTSTORE_DEFAULT_STORE": "local"
      }
    }
  }
}
```


```bash
objectstore-agent --mcp-url http://localhost:8000/mcp --host 0.0.0.0 --port 9001
```

<!-- BEGIN GENERATED: additional-deployment-options -->
### Additional Deployment Options

`objectstore-mcp` can run as a local stdio process or container, or behind a remote
network boundary. The
[Deployment guide](https://knuckles-team.github.io/objectstore-mcp/deployment/) carries
the detailed transport contract.

- **Local container** — launch a reviewed immutable image as a least-privilege
  stdio child with no listener or published port.
- **Remote URL** — connect through an operator-supplied authenticated HTTPS
  ingress. Keep its URL, outbound identity references, trust profile, and exact
  `MCP_ALLOWED_HOSTS` in `AgentConfig`.
<!-- END GENERATED: additional-deployment-options -->

## Docker deployment

```bash
docker compose -f docker/mcp.compose.yml up -d      # MCP server only
curl -s http://localhost:8000/health                 # {"status":"OK"}
```

Both services read configuration from `../.env` (copy
[`.env.example`](.env.example)); see [`docs/deployment.md`](docs/deployment.md).

## Development

```bash
pip install -e .[test]
pytest                       # full suite (cloud SDKs not required)
pip install -e .[test-s3]    # adds boto3+moto integration tests
pre-commit run --all-files
```

See [`docs/`](docs/) for architecture, concepts, and deployment details.

## License

MIT — see [LICENSE](LICENSE).



## Environment Variables

<!-- ENV-VARS-TABLE:START -->

#### Package environment variables

| Variable | Example | Description |
|----------|---------|-------------|
| `HOST` | `0.0.0.0` |  |
| `PORT` | `8000` |  |
| `TRANSPORT` | `stdio` | options: stdio, streamable-http, sse |
| `ENABLE_OTEL` | `True` |  |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:8080/api/public/otel` |  |
| `OTEL_EXPORTER_OTLP_PUBLIC_KEY` | secret-injected |  |
| `OTEL_EXPORTER_OTLP_SECRET_KEY` | secret-injected |  |
| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` |  |
| `EUNOMIA_TYPE` | `none` | options: none, embedded, remote |
| `EUNOMIA_POLICY_FILE` | `mcp_policies.json` |  |
| `EUNOMIA_REMOTE_URL` | `http://eunomia-server:8000` |  |
| `OBJECTSTORE_STORES` | `{"primary": {"backend": "s3", "bucket": "example-bucket", "endpoint": "https://object-store.example"}}` | Named stores (JSON). The zero-infra "local" filesystem store always exists. |
| `OBJECTSTORE_DEFAULT_STORE` | `local` |  |
| `OBJECTSTORE_FS_ROOT` | `~/.local/share/objectstore-mcp` |  |
| `OBJECTSTORE_MAX_GET_BYTES` | `10485760` | Safety limits (bytes / keys) |
| `OBJECTSTORE_MAX_PUT_BYTES` | `10485760` |  |
| `OBJECTSTORE_MAX_TRANSFER_BYTES` | `104857600` |  |
| `OBJECTSTORE_MAX_BATCH_KEYS` | `100` |  |
| `OBJECTSTORE_MAX_LIST_KEYS` | `1000` |  |
| `AZURE_STORAGE_CONNECTION_STRING` | — | Provider credentials resolve through each SDK's own chain (this package never reads them itself — the cloud SDK does): S3  : boto3 chain (AWS access key / secret / profile, ~/.aws, instance role) GCS : the gcloud application-default credentials chain (service-account file) Azure is the exception — the code reads this connection string directly: |
| `DEFAULT_AGENT_NAME` | `ObjectStore Agent` |  |
| `AGENT_DESCRIPTION` | `AI agent for object-storage operations.` |  |
| `AGENT_SYSTEM_PROMPT` | — |  |
| `MCP_URL` | `http://localhost:8000/mcp` |  |
| `OBJECTSTORETOOL` | `True` |  |

#### Inherited scion-sdk variables (apply to every connector)

| Variable | Example | Description |
|----------|---------|-------------|
| `MCP_TOOL_MODE` | `intent` | Tool surface: `intent` \| `condensed` \| `verbose` \| `both` |
| `MCP_ENABLED_TOOLS` | — | Comma-separated tool allow-list |
| `MCP_DISABLED_TOOLS` | — | Comma-separated tool deny-list |
| `MCP_ENABLED_TAGS` | — | Comma-separated tag allow-list |
| `MCP_DISABLED_TAGS` | — | Comma-separated tag deny-list |
| `MCP_CLIENT_AUTH` | — | Outbound MCP child auth: `oidc-client-credentials` \| `basic` \| `none` |
| `OIDC_CLIENT_ID` | — | OIDC client id (service-account auth) |
| `OIDC_CLIENT_SECRET_REF` | `secret://identity/oidc-client-secret` | Runtime secret reference for the OIDC service account |
| `MCP_BASIC_AUTH_USERNAME` | — | HTTP Basic username (`MCP_CLIENT_AUTH=basic`) |
| `MCP_BASIC_AUTH_PASSWORD_REF` | `secret://identity/mcp-basic-password` | Runtime secret reference for HTTP Basic auth (`MCP_CLIENT_AUTH=basic`) |
| `DEBUG` | `False` | Verbose logging |
| `PYTHONUNBUFFERED` | `1` | Unbuffered stdout (recommended in containers) |
| `PROVIDER` | `openai` | LLM provider for the agent |
| `MODEL_ID` | `gpt-4o` | Model id for the agent |
| `ENABLE_WEB_UI` | `True` | Serve the AG-UI web interface |

_27 package + 15 inherited variable(s). Auto-generated from `.env.example` + the shared scion-sdk set — do not edit._
<!-- ENV-VARS-TABLE:END -->

<!-- GOVERNED-CAPABILITY:START -->
## Governed capability contract

This package ships a compact canonical skill surface with specialist procedures
kept as referenced workflows. The current MCP tools, skill metadata,
`connector_manifest.yml`, ontology, mappings, shapes, fixtures, migrations,
tool-schema fingerprints, and certification metadata form one versioned
capability contract. Validate them together; do not rely on stale tool names or
historical per-task skill wrappers.

Runtime endpoints, credentials, certificate trust, tenant identity, retention,
and observability policy are deployment inputs and are never packaged values.
See [Configuration, trust, and privacy](docs/configuration.md) before enabling a
network transport, connector ingestion, Arbora delegation, or trace export.
<!-- GOVERNED-CAPABILITY:END -->
