Metadata-Version: 2.5
Name: CVE-RiskEngine-RF
Version: 0.1.0
Summary: Calibrated random forest scoring of CVE exploitability against disclosure-time EPSS, with the honest limits of the reference study attached.
Project-URL: Homepage, https://pypi.org/project/CVE-RiskEngine-RF/
Project-URL: Repository, https://github.com/rezayw/CVE-RiskEngine-RF
Project-URL: Paper, https://github.com/rezayw/CVE-RiskEngine-RF/tree/main/IEEEtranBST2
Author-email: Reza Yuzron Wardana <23525008@mahasiswa.itb.ac.id>, Hari Purnama <Hari.Purnama@itb.ac.id>
License: MIT License
        
        Copyright (c) 2026 Reza Yuzron Wardana and Hari Purnama
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: cisa-kev,cve,epss,exploitability,machine-learning,prioritization,random-forest,security-operations,vulnerability-management
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: >=3.10
Requires-Dist: joblib>=1.4
Requires-Dist: numpy>=1.26
Requires-Dist: scikit-learn<1.7,>=1.4
Provides-Extra: api
Requires-Dist: fastapi>=0.111; extra == 'api'
Requires-Dist: pydantic>=2.7; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.29; extra == 'api'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.2; extra == 'dev'
Requires-Dist: twine>=5.1; extra == 'dev'
Description-Content-Type: text/markdown

# cve-risk-engine

Calibrated random forest scoring of CVE exploitability, packaged with the honest
limits of the study it comes from.

This is the reference implementation of the scoring layer of

> **Exploitability-Driven CVE Prioritization for Security Operations: A Recursive
> Feature Elimination Random Forest Approach** — Reza Yuzron Wardana, Hari Purnama
> (ICDXA 2026). Code, data and evaluation artifacts:
> [CVE-RiskEngine-RF](https://github.com/rezayw/CVE-RiskEngine-RF)

**What this package is not.** It is not a claim that machine learning beats EPSS. On
the study's held-out temporal test partition the forest ranks better than
disclosure-time EPSS, but the difference in average precision is not statistically
established, the raw scores are not probabilities, and a CVSS-only score stays
close behind. The package ships those facts as metadata and refuses to return an
uncalibrated number as a probability.

## Install

```bash
pip install CVE-RiskEngine-RF
# optional HTTP service
pip install "CVE-RiskEngine-RF[api]"
```

## Use

```python
from cve_risk_engine import CveRiskEngine

engine = CveRiskEngine.load()
result = engine.score(
    cvss_vector="CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
    cvss_score=9.8,
    epss_score=0.00061,   # EPSS at or before the disclosure date, not today's value
    cve_id="CVE-2025-14611",
    cwe="CWE-798",
)
```

```python
{
  "cve_id": "CVE-2025-14611",
  "raw_model_score": 0.897937,
  "calibrated_probability": 0.012108,
  "cvss_score": 9.8,
  "epss_score": 0.00061,
  "final_risk_score": 0.4961,
  "risk_level": "medium",
  "recommendation": "Patch within 30 days",
  "features": {...}
}
```

Command line:

```bash
cve-risk-engine info
cve-risk-engine score --vector "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" \
                      --cvss 9.8 --epss 0.00061 --cwe CWE-798 --id CVE-2025-14611
cve-risk-engine batch records.csv --out scored.csv
```

HTTP:

```bash
uvicorn cve_risk_engine.api:app --port 8000
# GET /health  GET /model  POST /score  POST /score/batch
```

## Read the two numbers differently

| field | what it is | what it is not |
|---|---|---|
| `raw_model_score` | forest output, useful for **ranking** | a probability; Brier 0.0629 against 0.0019 for EPSS |
| `calibrated_probability` | Platt map fitted on the study's validation partition | a per-CVE certainty; the base rate is 0.245% |
| `final_risk_score` | `0.5 · calibrated + 0.5 · CVSS/10` | a fitted risk model; the weights are a policy choice |
| `risk_level` | band derived from `final_risk_score` | see the banding limitation below |

## What the model actually does

Corpus: 44,474 CVEs published in 2025 with a CVSS v3.1 vector. Target: CISA KEV
entry **within 90 days of disclosure**, 109 positives (0.245%); CVEs already
KEV-listed at disclosure are excluded, so no record is right-censored. Temporal
split: train 25,951 (70 positives) to 2025-07-31, validation 7,363 (18) Aug–Sep,
test 11,160 (21) Oct–Dec. Features: 7 of 13 candidates retained by RFECV on the
training partition. Imbalance: SMOTE 1:1 chosen on validation, applied inside the
training folds only.

| Metric (test) | Random forest | Disclosure-time EPSS | Paired Δ, 95% CI |
|---|---:|---:|---:|
| ROC-AUC | 0.8807 | 0.7833 | — |
| Average precision | 0.0159 | 0.0074 | +0.0085 [−0.0029, +0.0409], p=0.128 |
| MCC | 0.0589 | 0.0439 | +0.0150 [−0.0102, +0.0400], p=0.248 |
| Recall (validation threshold) | 0.9048 (queue 3,228) | 0.4762 (queue 1,483) | +0.4286 [+0.2222, +0.6364] |
| Brier, raw | 0.0629 | 0.0019 | +0.0610 [+0.0576, +0.0644] |
| Brier, Platt-calibrated | 0.00187 | 0.0019 | — |

Ranking differences on average precision and MCC are inside their bootstrap
intervals. The recall gain is real but comes at roughly twice the queue length.

## KEV membership is not a risk band, and high/critical are unreachable

A natural question is whether a CVE that is listed in CISA KEV comes back as `high`
or `critical`. It does **not**, and the engine never accepts KEV as an input: KEV is
the evaluation label of the study, never a feature, so a served decision is
reproducible from the model alone. On the test partition the 21 KEV-listed CVEs come
back as **15 `medium` and 6 `low`**, none above `medium`, with `final_risk_score`
between 0.3058 and 0.5061. A KEV CVE at CVSS 7.5 lands in `low`; so does one at CVSS
6.1.

The reason is the one in the second limitation below: at a 0.245% base rate,
`calibrated_probability` is at most a few percent, so `FinalRisk` is dominated by the
CVSS term and saturates around 0.53. Read the band as a severity-flavoured label, not
as an exploitation verdict, and never as a KEV oracle. Rank by `final_risk_score` and
cut by queue capacity; that is the operating point the study evaluates.

## Two limitations you must design around

**1. Raw scores are not probabilities.** They are ranked correctly but are badly
calibrated (Brier 0.0629). Always read `calibrated_probability`, never
`raw_model_score`, when a number is shown to a human.

**2. The `high` and `critical` bands are unreachable.** The bounds come from the
CVSS v3.1 severity scale on the unit interval, but once the score is calibrated on
a 0.245% base rate, `final_risk_score` cannot exceed about 0.53 even at CVSS 10.0.
On the test partition the `high` and `critical` bands hold zero records; everything
lands in `low` (8,421 records, 6 exploited) or `medium` (2,739 records, 15
exploited). Precision still rises monotonically between them, but a banding scheme
usable under calibrated probabilities has to be defined on **validation quantiles
of the score**, not on the CVSS scale. For that reason the metadata ships
`capacity_cutoffs_validation` (the score above which the top 0.5%, 1%, 2%, 5% and
10% of the validation backlog fall) and the package README recommends cutting by
capacity. Treating `risk_level` as an absolute severity rating is wrong.

Blended-score sensitivity on the calibrated probability, for reference: test
average precision rises with the model weight (0.0295 at 0.5/0.5, 0.0331 at
0.7/0.3, 0.0347 at 0.8/0.2) against 0.0159 for the model alone and 0.0102 for CVSS
alone, while the operational cost of a 500-record queue is identical (1,590, with 10
of the 21 exploited CVEs caught) for every blend from 0.5 upward. Equal weights are
used because they sit in the middle of that flat region.

## Bundled artifact

`cve_risk_engine/artifacts/` holds the classifier, the Platt map, the CWE
frequency map fitted on the training partition, and a metadata file recording the
hyperparameters, the target definition, the split windows, the test metrics, the
paired intervals, the input SHA-256 digests and the limitation above. The artifact
reproduces the manuscript's test numbers exactly (average precision 0.015903,
ROC-AUC 0.880724), and `tests/reference_scores.csv` freezes 821 test records
including all 21 positives so anyone can re-verify that claim without the study
repo:

```bash
python -m pytest tests -q
```

## Rebuilding the artifact

The artifact is exported from the study pipeline, never hand-edited:

```bash
python scripts/export_artifact.py \
    --study-repo /path/to/CVE-RiskEngine-RF \
    --out src/cve_risk_engine/artifacts
python scripts/build_reference_scores.py \
    --study-repo /path/to/CVE-RiskEngine-RF \
    --out tests/reference_scores.csv
```

The first script asserts that the exported model's test average precision and
ROC-AUC are bit-identical to `revision/outputs/results.json`; if they drift, the
export fails rather than shipping a lookalike.

## Licence

MIT. The bundled artifact is trained on derived metadata from NVD, CISA and FIRST;
the upstream terms of those sources apply to the data.
