Metadata-Version: 2.4
Name: datahub-describer
Version: 0.1.0
Summary: Hierarchical LLM-generated descriptions for DataHub
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: acryl-datahub[datahub-rest]>=1.7.0
Requires-Dist: openai>=1.40.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# DataHub Describer

Generate missing DataHub descriptions with an OpenAI-compatible LLM.

The connector works from fields upward:

```text
fields → datasets → parent containers
```

Field prompts use field metadata. Parent prompts use the names and descriptions
of their immediate children. Existing descriptions are preserved by default.

## Install

```bash
pip install datahub-describer
```

## Configure

Use the registered DataHub source type:

```yaml
source:
  type: datahub-describer
  config:
    llm_model: "${LLM_MODEL}"
    llm_base_url: "${LLM_BASE_URL}"
    llm_api_key: "${LLM_API_KEY}"

sink:
  type: datahub-rest
  config:
    server: "${DATAHUB_GMS_URL}"
```

See `datahub_describer.example.yml` for all settings.

## Development

```bash
pip install -e '.[dev]'
pytest
```
