Metadata-Version: 2.5
Name: devoriz
Version: 0.3.0
Summary: Read-only production triage: digest your logs, model your flows, and give an agent a safe way to investigate.
Project-URL: Homepage, https://github.com/ameerfayiz/devoriz
Project-URL: Documentation, https://github.com/ameerfayiz/devoriz/tree/main/docs
Project-URL: Issues, https://github.com/ameerfayiz/devoriz/issues
Project-URL: Changelog, https://github.com/ameerfayiz/devoriz/blob/main/CHANGELOG.md
Project-URL: Source, https://github.com/ameerfayiz/devoriz
License: Apache-2.0
License-File: LICENSE
Keywords: agent,elasticsearch,grafana,incident-response,kafka,llm,loki,observability,pii,redaction,sre,triage
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Requires-Dist: pyyaml<8,>=6
Provides-Extra: agent
Requires-Dist: anthropic<1,>=0.79; extra == 'agent'
Requires-Dist: cryptography<51,>=42; extra == 'agent'
Requires-Dist: fastapi<1,>=0.110; extra == 'agent'
Requires-Dist: pydantic<3,>=2; extra == 'agent'
Requires-Dist: ruamel-yaml<0.20,>=0.18; extra == 'agent'
Requires-Dist: uvicorn<1,>=0.27; extra == 'agent'
Provides-Extra: all
Requires-Dist: anthropic<1,>=0.79; extra == 'all'
Requires-Dist: cryptography<51,>=42; extra == 'all'
Requires-Dist: fastapi<1,>=0.110; extra == 'all'
Requires-Dist: mcp<2,>=1.0; extra == 'all'
Requires-Dist: pydantic<3,>=2; extra == 'all'
Requires-Dist: pypdf<7,>=4; extra == 'all'
Requires-Dist: ruamel-yaml<0.20,>=0.18; extra == 'all'
Requires-Dist: uvicorn<1,>=0.27; extra == 'all'
Provides-Extra: dev
Requires-Dist: coverage[toml]>=7; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.0; extra == 'mcp'
Provides-Extra: pdf
Requires-Dist: pypdf<7,>=4; extra == 'pdf'
Provides-Extra: share
Requires-Dist: cryptography<51,>=42; extra == 'share'
Description-Content-Type: text/markdown

# Devoriz

[![CI](https://github.com/ameerfayiz/devoriz/actions/workflows/ci.yml/badge.svg)](https://github.com/ameerfayiz/devoriz/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/devoriz.svg)](https://pypi.org/project/devoriz/)
[![Python](https://img.shields.io/pypi/pyversions/devoriz.svg)](https://pypi.org/project/devoriz/)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)


> **De·vo·riz** /dɛˈvɔːrɪz/
> *devour* — to consume at volume, without ceremony
> *-ize* — the morpheme of becoming structured
>
> **It eats the incident. It returns the lesson.**

Devoriz is read-only production triage: digest your logs, model your flows,
and give an agent a safe way to investigate.

Most tooling stops at root cause — the trace, the diff, the failing
assertion. That's the easy half. The hard half is that the same class of
failure returns in six weeks wearing different clothes, and the memory of
how you fixed it left with the engineer who fixed it.

Devoriz reads the logs, reads the code, finds the fault — then reconstructs
the flow that led there and writes the lesson back into something durable. 

The incident is metabolized, not just closed.

### Quick start

```bash
pip install devoriz
devoriz init myapp                         # scaffold a profile for your platform
devoriz doctor                             # check it, without querying anything
devoriz inventory && devoriz map           # discover environments and targets
devoriz logs errors -e prod -t api -s 2h   # a digest, not a wall of lines
devoriz ui                                 # the same tools, driven by an agent
```

Or skip the first four: `pip install 'devoriz[agent]' && devoriz ui` opens a
settings screen that creates the profile, fills in the environments and
credentials, edits the redaction policy and runs the same checks — and a **Docs**
screen with every guide in it, starting from
[getting-started](docs/getting-started.md).

`diz` is installed as a shorter alias for the same command — the docs use
`devoriz` throughout, but nobody types seven characters at 3am.

Point it at Grafana/Loki, Elasticsearch and Kafka — or at whatever you run, via a
plugin — and describe your platform once in a **profile**. Everything after that
is generic.

---

## Why this exists

Four problems, each of which has a cheap answer that almost nobody implements:

**Logs are unreadable at incident volume.** 50,000 lines is not a thing a human
or a model can read. Every command here defaults to a *fingerprint digest*:
lines collapsed by message template, with counts, a window and one exemplar. A
digest of 50,000 lines costs about what a digest of 200 costs, so you start with
the shape of the problem and drill into exactly one template.

```
 count  window        service                  template
  4812  13:20→14:05   api                      failed to resolve customer: <NUM>
    31  13:22→13:24   workers                  Cannot read properties of undefined (reading '<WORD>')
```

**"Where did it stop?" is a different question from "what is erroring?"** A
digest cannot answer it. `flow` is an executable model of how one entity moves
through your platform — the stages, the cheapest observable that proves each one,
the lag budget between them — and `probe` walks it, anchoring each query on the
*previous* stage's last line. Seven stages cost about fifteen lines of output
whether the services logged 200 lines that hour or 200,000.

```
✓ checkout       08-11 17:38:05.800           20 lines  api
✓ payment        08-11 17:38:06.848 +1s       13 lines  workers
✗ confirmation   nothing in 17:38:06->17:53:06  workers  [optional]

STOPPED after payment (last seen 17:38:06.848) — confirmation never saw it within 15m
  if: payment succeeded but no email
      look for: the template id, or a bounce in the provider webhook log
```

The model corrects itself: a stage that was seen climbs `assumed → likely →
confirmed`, a probe that worked is tried first next time, and a stage that missed
while a *later* one hit is marked `refuted` rather than quietly believed.

**Investigations forget everything.** Notes go stale silently, so nobody trusts
them, so nobody writes them. Here a note is a *belief* with a standing —
`confirmed` > `likely` > `assumed`, `~` when nobody has re-checked it in weeks,
`refuted` when it was disproved — and settling one is a single cheap call at the
end of a turn. Refuted notes are kept, because "we checked, it is not that" is
the cheapest thing memory can say.

**You cannot paste production logs into a model.** Not if they contain customers.
Every identifier is replaced by a **keyed pseudonym** before it reaches an LLM, a
terminal or a ticket:

```
9876543210          ->  <PHONE:7QK3M2>         9864563210  ->  <PHONE:X4BN0R>
+91 98765 43210     ->  <PHONE:7QK3M2>         same number, same token
jane.doe@gmail.com  ->  <EMAIL:J5J8QP>
ops@example.com     ->  <EMAIL:07QWK4>@example.com    staff domain survives
```

A partial mask (`98****210`) makes those two numbers indistinguishable and
destroys the most useful join in an investigation. `<phone>` is worse. A
pseudonym is **distinguishing** (different values, different tokens),
**correlating** (same value, same token, in every backend and every future
session) and **one-way** (HMAC-SHA256 under a key that never leaves the machine).

---

## Read-only by construction

Not by convention, and not by asking the model nicely:

- The HTTP layer refuses any method but GET unless the caller is a login handler,
  so producing a Kafka message or resetting an offset is unreachable code.
- Elasticsearch paths are validated per **segment** against a read allowlist, so
  `logs-x/_search/../_bulk` is refused rather than forwarded.
- The agent has no shell, no `kubectl`, and no write tool. The only things it can
  change are its own notes and its own flow models, both through a schema.
- `read_file` and `search_code` are confined to the configured repo roots by
  canonical-path check.
- `--no-mask` exists for an engineer at a terminal, relaxes only to
  *secrets-only* (no flag prints a bearer token), and is refused to the agent
  outright.
- **A pseudonym may never be searched for.** The token is minted here; every
  backend stores the real value, so searching for one returns zero rows — and a
  well-formed query returning zero rows reads as *the service never saw them*.
  Every search term is checked before it leaves.

---

## Configure it for your infra

One directory describes one platform. Switch platforms with `--profile`.

```
~/.devoriz/
  devoriz.yaml                   default profile, shared settings, plugins
  plugins/*.py                   drop-in plugins, no packaging required
  profiles/myapp/
    profile.yaml                 environments, credentials as ${VAR}, discovery rules, repos
    .env                         the secrets those vars resolve to (chmod 600)
    prompt.md                    what the agent needs to know about your platform
    pii.yaml                     what counts as identifying here
    skills/*.md                  how *you* triage a specific situation
    flows/*.yaml                 executable models, grown by probing
    memory/*.md                  what past investigations proved
    plugins/*.py                 backends only this platform has; travels with it
    .state/                      generated: targets, inventory, cookies, history, keys
```

`profile.yaml` never holds a secret — it names them:

```yaml
environments:
  prod:
    loki:
      url: https://grafana.example.com
      user: ${GRAFANA_USER}
      password: ${GRAFANA_PASSWORD}
    kafka:
      url: https://kafka-ui.example.com
      user: ${KAFKA_UI_USER}
      password: ${KAFKA_UI_PASSWORD}
```

Targets are discovered rather than typed: `devoriz inventory` sweeps every
datasource once and caches the label schema; `devoriz map` proposes environments
and named targets from it and asks you to confirm. Where your naming is unusual,
`discovery:` rules teach the mapper — regions, variants, how pod names collapse
into components, which host prefix is which tier. Hand-written entries in
`profile.yaml` always win over generated ones, so a fix is permanent.

`devoriz doctor` reports what is configured, what is missing and what would be
tried — without running a single query.

None of this needs an editor. `devoriz ui` → **Settings** is a form over the same
files: profiles (create, copy, switch, set the default, remove from the list
without deleting anything), every environment and backend block, the credentials
behind each `${VAR}`, the agent's prompt and skills, MCP servers, and the
redaction policy with a live tester. Every write is validated before it lands and
backed up after, and the raw YAML is always one click away.

---

## Extend it

Everything is a plugin, including the parts that ship in the box. `devoriz.backends`
and `devoriz.cli.builtin` register themselves through the same public API a
third party uses, which is how the API stays honest.

A plugin is any module with `register(registry)`, found three ways: a `.py` file
in `<workspace>/plugins/`, a module named in `plugins:` in your profile, or a
`devoriz.plugins` entry point in an installed package.

```python
from devoriz.plugins import BackendSpec

def register(reg):
    reg.add_backend(BackendSpec(
        name="datadog", summary="Datadog Logs", section="datadog",
        factory=lambda cfg, target=None, verbose=False: DatadogSource(cfg),
        add_parser=build_parser, probeable=True,
        agent_help="Datadog Logs. Subcommands: errors, search, trace.",
    ))
```

That one registration adds `devoriz datadog …` to the CLI, makes Datadog-backed
targets probeable by `flow`, and tells the agent the command exists — because the
tool list and the system prompt are both assembled from the registry rather than
written down. Six kinds of contribution:

| Kind | Adds |
|---|---|
| `BackendSpec` | a data source: a CLI command, a `flow` probe target, an agent tool value |
| `CommandSpec` | a CLI subcommand that is not a data source |
| `AgentToolSpec` | a typed tool the model calls directly |
| `SkillSpec` | a procedure in markdown — **no Python at all** |
| `DecoderSpec` | how to parse your house log format |
| `DetectorSpec` | an identifier shape the redactor should recognise |

**Skills are the cheapest extension point.** A markdown file with a `when:` line
in `<profile>/skills/` encodes how your team triages one situation. Always-on
skills are inlined in the prompt; the rest are listed by trigger and pulled with
`load_skill` when they apply, so a hundred skills stay affordable.

```markdown
---
name: stuck-consumer
when: a queue is not draining, or lag is climbing
---
1. `devoriz kafka lag -e <env>` — STABLE with climbing lag means stuck, not crash-looping…
```

See [docs/plugins.md](docs/plugins.md), [docs/skills.md](docs/skills.md) and
[examples/plugins/](examples/plugins/).

---

## The agent

`devoriz ui` opens a chat UI backed by an agent that drives the same commands you
do, asks clarifying questions when the answer depends on them, and writes down
what it learns.

A message can carry more than words. Drop a file on the window, paste a
screenshot, or use ＋: a text file is inlined redacted — head first, with
`read_attachment` to page or grep the rest — while an image goes as it is, which
the composer warns you about because nothing can pseudonymise the inside of a
screenshot. A **document** — PDF, `.docx`, `.pptx`, `.xlsx`, OpenDocument — is
converted to text on your own machine first, so the incident report full of
customers goes through the same gate every log line does. (Office formats need
nothing installed; PDF uses `pypdf` or `pdftotext`, and a scan neither can read
is handed over whole, flagged as unredacted.) And `@` completes against your
profile: `@api` becomes
`repo="api"` for the code tools, `@prod` becomes `-e prod` for every query,
aliases included, so the agent never spends a turn asking which environment you
meant.

No API key required: it uses the OAuth token Claude Code already stored. That
token shares a rate limit with your interactive session, so set
`ANTHROPIC_API_KEY` for an independent quota. MCP servers (Jira, GitHub, …)
configured in `<profile>/mcp.json` become tools with the same redaction gate over
their results — a ticket describing a customer's problem carries identifiers no
log rule would ever see.

Recall and persistence are one loop:

```
1. RECALL      memory_search → index lines ranked by how well proved
2. INVESTIGATE logs · kafka · docs · search_code · flow probe
3. SETTLE      memory_observe confirmed → climbs, clock reset
                              refuted   → dead, kept, unranked
4. WRITE       memory_write / flow_write — the corrected claim
5. ANSWER      a bounded persist follow-up fires only if the turn owed memory something
```

---

## Install

```bash
pip install devoriz            # the CLI: PyYAML only
pip install 'devoriz[all]'     # plus the agent, its web UI and profile sharing
```

The extras are opt-in, so `pip install devoriz` really does install nothing but
PyYAML — `devoriz[agent]` is what brings FastAPI and uvicorn for `devoriz ui`, and
`devoriz[share]` the cipher for `devoriz share`.

Python 3.10+. No services to run, no database, no daemon.

## Tests

```bash
tests/run.sh           # offline: no network, no credentials, no API quota
devoriz pii self-test  # the redaction corpus against the policy you installed
```

Each suite runs in its own process, because they install different workspaces and
different redaction policies. `pytest tests/t_plugins.py` works for a single one.

## Documentation

Every guide below also ships **inside the package**: `devoriz ui` → **Docs**
renders them, searches them, and links between them — so a pip install carries
its own manual.

| | |
|---|---|
| [docs/getting-started.md](docs/getting-started.md) | zero to a shared team profile, in eleven stages |
| [docs/configuration.md](docs/configuration.md) | profiles, environments, targets, discovery rules |
| [docs/settings.md](docs/settings.md) | configuring all of it from the browser |
| [docs/sharing.md](docs/sharing.md) | sending a profile to a colleague as one sealed file |
| [docs/plugins.md](docs/plugins.md) | adding a backend, a command, a tool |
| [docs/skills.md](docs/skills.md) | teaching it how your team triages |
| [docs/flows.md](docs/flows.md) | building an executable model of a journey |
| [docs/pii.md](docs/pii.md) | the redaction policy, and what it does not claim |
| [docs/architecture.md](docs/architecture.md) | ports, adapters, and where to put a change |
| [docs/migrating.md](docs/migrating.md) | porting a single-platform fork onto profiles |

## Contributing

Issues and pull requests are welcome. The full guide is in
[CONTRIBUTING.md](CONTRIBUTING.md); the short version:

```bash
git clone https://github.com/ameerfayiz/devoriz && cd devoriz
python -m venv .venv && source .venv/bin/activate
pip install -e ".[agent,dev]"

tests/run.sh                 # must be green
ruff check src tests         # must be clean — CI fails on any violation
```

**The cheapest contribution is not Python.** A skill is a markdown file
describing how you triage one situation, and a detector is an identifier shape —
both are useful to everyone and neither needs a code change. See
[docs/skills.md](docs/skills.md) and [docs/pii.md](docs/pii.md).

Two things in this codebase are load-bearing rather than stylistic, and a change
that touches either needs to say so in the pull request:

- **Nothing writes.** Non-GET is unreachable from the query path. If a change
  needs a POST to a backend, open an issue before a patch.
- **Nothing identifying escapes.** Every path to a model, a terminal, a file or a
  ticket goes through `scrub()`. A new detector comes with two corpus cases in
  `t_pii.py` — one that must be redacted, and one that must survive, because
  over-redaction destroys the joins an investigation runs on.

CI runs the suite on Python 3.10–3.13, installs the built wheel and drives it end
to end, and fails if a `.env` or `.state/` path is ever committed.

Found a security issue — a redaction bypass, a way to make it write, a
confinement escape? **Do not open an issue.**
[Report it privately](https://github.com/ameerfayiz/devoriz/security/advisories/new);
see [SECURITY.md](SECURITY.md) for what is in scope and what to include. By
participating you agree to the [Code of Conduct](CODE_OF_CONDUCT.md).

## License

Apache-2.0.
