Metadata-Version: 2.5
Name: elementary-runtime
Version: 0.1.0rc4
Summary: Customer-side runtime that executes Elementary Cloud SQL against a warehouse.
Project-URL: Homepage, https://www.elementary-data.com
Project-URL: Repository, https://github.com/elementary-data/elementary-runtime
Author-email: Elementary Data <team@elementary-data.com>
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <3.15,>=3.10
Requires-Dist: expandvars<2,>=1.1.2
Requires-Dist: httpx<1,>=0.28.1
Requires-Dist: pandas<4,>=2.0
Requires-Dist: pydantic<3,>=2.11
Requires-Dist: pyyaml<7,>=6
Requires-Dist: sqlalchemy~=2.0
Requires-Dist: sqlglot[c]~=30.4
Requires-Dist: tenacity<10,>=8.0
Requires-Dist: typing-extensions<5,>=4.7
Provides-Extra: databricks
Requires-Dist: databricks-sdk<1,>=0.30; extra == 'databricks'
Requires-Dist: databricks-sqlalchemy~=2.0; extra == 'databricks'
Provides-Extra: postgres
Requires-Dist: psycopg2-binary<3,>=2.9; extra == 'postgres'
Provides-Extra: redshift
Requires-Dist: psycopg2-binary<3,>=2.9; extra == 'redshift'
Requires-Dist: sqlalchemy-redshift~=1.0; extra == 'redshift'
Provides-Extra: snowflake
Requires-Dist: cryptography<51,>=42.0; extra == 'snowflake'
Requires-Dist: snowflake-sqlalchemy~=1.8; extra == 'snowflake'
Description-Content-Type: text/markdown

# Elementary Runtime

> Early development.

Elementary Runtime is a small service that customers run in their own
environment. It uses local warehouse credentials to execute SQL prepared by
Elementary Cloud, so Cloud never holds credentials that can read customer
business tables.

Warehouse drivers will be optional extras. Installing the base package for
task models will not pull in every warehouse driver or the full runtime.

A license has not been chosen yet.

## Security-critical code

Local safety controls live under
[`src/elementary_runtime/execution/`](src/elementary_runtime/execution/).
That tree is the audited surface: SQL validation, query execution with limits
applied while rows are fetched and encoded, and redaction of SQL, row values,
credentials, and unrestricted warehouse errors from logs and telemetry.

## Repository layout

```text
src/elementary_runtime/
  protocol/      Task and result wire formats
  config/        Versioned YAML configuration and local limits
  common/        Shared helpers used by protocol and config
  warehouses/    Warehouse clients (drivers behind extras)
  execution/     Audited local safety controls
    sql/         SQL validation
    worker/      Worker pool, query execution, limit enforcement
    redaction/   Error, log, and telemetry scrubbing
  coordinator/   Register, task streams, heartbeat, submit, retries, shutdown
  cli/           Process entrypoint: commands, instance id, logging
```

## Configuration

```yaml
version: 1
cloud:
  token_file: /run/secrets/elementary_runtime_token
connections:
  - warehouse_connection_id: connection-id
    type: snowflake
    account: ab12345.eu-central-1
    user: elementary_runtime
    password_file: /run/secrets/snowflake_password
    role: ELEMENTARY_RUNTIME
    warehouse: ELEMENTARY_WH
    database: ANALYTICS
```

## Running

`elementary-runtime` loads the YAML configuration and runs until it is stopped.

```bash
elementary-runtime run --config /etc/elementary-runtime/config.yml
elementary-runtime validate --config /etc/elementary-runtime/config.yml
elementary-runtime validate --check-connections --config /etc/elementary-runtime/config.yml
elementary-runtime --version
```

A container can start with no arguments. Each flag falls back to an environment variable:

- `ELEMENTARY_RUNTIME_CONFIG`: configuration file, default `/etc/elementary-runtime/config.yml`
- `ELEMENTARY_RUNTIME_INSTANCE_ID`: explicit instance id, unset by default
- `ELEMENTARY_RUNTIME_LOG_LEVEL`: root log level, default `INFO`
- `ELEMENTARY_RUNTIME_POD_UID`: Helm pod UID from the Downward API
- `ELEMENTARY_RUNTIME_POD_NAME`: pod name, used as the display name when the config has none

The instance id is the first match:

1. `--instance-id` or `ELEMENTARY_RUNTIME_INSTANCE_ID`, used as given.
2. `helm:<pod uid>` when `ELEMENTARY_RUNTIME_POD_UID` is set. `instance.id_prefix` is not applied, because a pod UID is already unique.
3. `docker:<hostname>` when `/.dockerenv` or `/run/.containerenv` exists, otherwise `host:<hostname>`. `instance.id_prefix` is prepended when set, for example `eu-prod:docker:runtime-1`.

Docker assigns a new hostname when a container is recreated. Set one explicitly (`docker run --hostname` or Compose `hostname`) so the instance id stays stable, and keep it unique among replicas.

Exit codes:

- `0`: stopped gracefully
- `1`: runtime failure, such as Cloud rejecting the token
- `2`: usage or configuration error
- `3`: drain timed out with work still in flight

The first SIGTERM or SIGINT drains in-flight work for 30 seconds. A second signal exits immediately. Set the container stop grace period above that drain (`docker stop -t`, or Compose `stop_grace_period`), for example 45 seconds.

## Docker images

Published images live at `ghcr.io/elementary-data/elementary-runtime` and are built for `linux/amd64` and `linux/arm64`.

| Tag | Drivers |
| --- | --- |
| `{version}` | All of them |
| `latest` | All of them, stable releases only |
| `core-{version}` | None |
| `core` | None, stable releases only |
| `{driver}-{version}` | `snowflake`, `databricks`, `postgres`, or `redshift` |
| `{driver}` | That driver, stable releases only |

A pre-release such as `0.1.0rc3` is tagged with its version only. It does not move `latest`, `core`, or the bare driver tags.

```bash
docker run --rm \
  --hostname runtime-1 \
  --stop-timeout 45 \
  -v /path/to/config.yml:/etc/elementary-runtime/config.yml:ro \
  ghcr.io/elementary-data/elementary-runtime:{version}
```

The image runs `elementary-runtime run`. Mount the configuration where the process expects it, set `--hostname` so the instance id stays stable, and keep the stop timeout above the 30 second drain.

## Development

Requires [uv](https://docs.astral.sh/uv/) and Python 3.10–3.14. Local
development uses Python 3.12.

After a clone or a new Git worktree, install the environment and hooks:

```bash
./script/setup.sh
```

Then:

```bash
uv run pytest
uv run mypy src tests
uvx prek run --all-files
```

The package version lives in `pyproject.toml`. At runtime,
`elementary_runtime.__version__` is read from the installed distribution
metadata.

## Releasing

Run the Prepare release workflow from the Actions tab. Choose a bump (`patch`, `minor`, `major`, `rc`, `stable`, `patch-rc`, `minor-rc`, or `major-rc`) or set an explicit version. `minor-rc` turns `0.1.0` into `0.2.0rc1`. `rc` advances an existing candidate. It opens a pull request that updates `pyproject.toml` and `uv.lock`.

Merging that pull request publishes the package to PyPI and the container images, then tags `v{version}` and creates a GitHub Release. Pre-releases do not move floating image tags.

If publishing fails before that tag exists, re-run the Release package workflow from `main`. A version that already has a tag is skipped.
