Metadata-Version: 2.4
Name: mlflow-drift-plugin
Version: 0.1.0
Summary: MLflow plugin for detecting feature and prediction drift using PSI, KS-test, and chi-square.
Home-page: https://github.com/khushi-070906/mlflow-drift-plugin
Author: Khushi Mittal
Project-URL: Homepage, https://khushi-070906.github.io/mlflow-drift-plugin/
Project-URL: Source, https://github.com/khushi-070906/mlflow-drift-plugin
Project-URL: Issues, https://github.com/khushi-070906/mlflow-drift-plugin/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.23
Requires-Dist: scipy>=1.9
Provides-Extra: mlflow
Requires-Dist: mlflow>=2.0; extra == "mlflow"
Provides-Extra: yaml
Requires-Dist: pyyaml>=6.0; extra == "yaml"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: mlflow>=2.0; extra == "dev"
Requires-Dist: pyyaml>=6.0; extra == "dev"
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# mlflow-drift-plugin

Detect feature and prediction drift between a baseline dataset and new data — as a lightweight MLflow plugin.

Most drift-detection tools either require a hosted platform or bolt monitoring on after the fact. This plugin plugs into MLflow, which you're likely already using to track experiments, so drift checks become part of your existing run lifecycle instead of a separate system to maintain.

## What it does

- Compares a **baseline** dataset (e.g. training data, or a designated reference run) against **current** data (e.g. a new validation batch or production sample)
- Runs the right statistical test per column automatically:
  - Numeric columns → **PSI** (Population Stability Index) + **KS-test**
  - Categorical columns → **PSI** + **chi-square**
- Produces a JSON report (for CI/automation) and a self-contained HTML dashboard (for humans)
- Optionally attaches both as artifacts on the active MLflow run, and can fail the run if drift is detected (`fail_on_drift=True`)

## Install

```bash
pip install -e .              # core (pandas, numpy, scipy)
pip install -e ".[mlflow]"    # + MLflow integration
pip install -e ".[dev]"       # + test dependencies
```

## Quick start — no MLflow required

```python
from drift_plugin import DriftDetector

detector = DriftDetector(psi_threshold=0.2, p_value_threshold=0.05)
detector.set_baseline(baseline_df)
report = detector.detect(current_df)

print(report.overall_drifted)     # True/False
print(report.drifted_features)    # ['applicant_income', 'region']
print(report.to_json())
```

## Quick start — with MLflow

```python
import mlflow
from drift_plugin import check_run_for_drift, DriftConfig

with mlflow.start_run():
    # ... train and log your model as usual ...
    check_run_for_drift(
        current_df=validation_df,
        baseline_run_id="abc123",           # a previous run with a logged dataset artifact
        config=DriftConfig(fail_on_drift=True),
    )
```

This attaches `drift_report.json` and `drift_report.html` to the run's artifacts, logs a `drift_features_flagged` metric, and tags the run `drift_status=drifted|stable`.

### Automatic checking on every run

```python
from drift_plugin.plugin_hooks import enable_drift_autolog

enable_drift_autolog(
    get_current_df=lambda: latest_validation_batch(),
    baseline_run_id="abc123",
)
# every mlflow.end_run() from here on runs a drift check automatically
```

## Configuration

```python
from drift_plugin import DriftConfig

config = DriftConfig(
    psi_threshold=0.2,           # PSI >= this = drifted
    p_value_threshold=0.05,      # p-value < this = drifted
    monitored_columns=["applicant_income", "region"],  # None = check all shared columns
    ignored_columns=["id", "timestamp"],
    fail_on_drift=True,          # raise DriftDetectedError if drift found
)
```

Or load from file: `DriftConfig.from_file("drift_config.yaml")` / `.json`.

## Compliance layer (RBI draft MRM / DPDP-oriented)

Optional add-on: wraps a `DriftReport` in language and framing relevant to Indian financial-sector model risk management.

```python
from drift_plugin import build_compliance_report, write_compliance_report

compliance_report = build_compliance_report(
    drift_report,
    model_id="loan-approval-v3",
    regulated_entity_name="Example NBFC Pvt Ltd",
    personal_data_features=["applicant_income"],  # flags DPDP relevance in the narrative
)
write_compliance_report(compliance_report, json_path="compliance.json", html_path="compliance.html")
```

**Read this before using it for anything real:** as of August 2026, the RBI's *"Guidance on Regulatory Principles for Model Risk Management, 2026"* (released 24 June 2026) is a **draft** — the comment period closed 24 July 2026, but RBI hasn't issued final guidance or an implementation date. This module maps drift findings to that draft's structure (it names "data risks," including drift, as one of seven AI risk dimensions entities must test for and document) because it's the most concrete public signal available, not because it's binding law today. DPDP Act Sec. 8(3)-(4) is cited as a forward-looking hook; its substantive obligations aren't enforceable until 13 May 2027.

Every report this module generates — JSON and HTML — carries this disclaimer inline, not just in this README, because the artifact itself is what would end up in front of a compliance officer or auditor. **This is not legal advice.** Have counsel or an internal compliance team review the mapping before it goes anywhere near an actual regulatory submission, and revisit it once RBI issues the final guidance.

## Repo structure

```
drift_plugin/
├── detector.py       # core PSI / KS-test / chi-square logic
├── baseline.py       # loads baseline data from a DataFrame or an MLflow run artifact
├── plugin_hooks.py   # MLflow integration (check_run_for_drift, autolog)
├── report.py         # JSON + HTML report rendering
├── compliance.py     # RBI draft MRM / DPDP-oriented compliance report (optional)
└── config.py         # thresholds and monitoring settings
tests/                 # pytest unit tests
examples/
└── demo_notebook.ipynb  # synthetic dataset with injected drift, end to end
```

## Demo

Open `examples/demo_notebook.ipynb` — it builds a synthetic loan-approval dataset, injects a deliberate distribution shift, runs detection, and generates an HTML report you can open directly in a browser.

## Running tests

```bash
pip install -e ".[dev]"
pytest tests/
```

## Why PSI + KS-test / chi-square, not something fancier

PSI is the industry-standard metric for population stability in regulated/financial contexts (widely used in credit risk model monitoring) — it's interpretable to non-technical reviewers (compliance, risk teams), not just data scientists. KS-test and chi-square add statistical significance on top of PSI's magnitude-only signal. This combination is deliberately simple and well-understood rather than a black-box drift score, which matters if you ever need to explain a flagged drift event to someone outside engineering.

## Status

Early / v0.1.0. Core detection is stable and tested. MLflow integration covers explicit and autolog-style usage; a full setuptools entry-point registration (so MLflow discovers the plugin without any import) is planned once there's real usage to justify it.

## License

MIT
