Metadata-Version: 2.5
Name: semanticmemo
Version: 2.0.1
Summary: Production-grade semantic caching for LLM agents — a learned equivalence classifier replaces the naive cosine threshold that causes false-positive failures.
Project-URL: Homepage, https://github.com/rajveer100704/semanticmemo
Project-URL: Repository, https://github.com/rajveer100704/semanticmemo
Project-URL: Issues, https://github.com/rajveer100704/semanticmemo/issues
Project-URL: Changelog, https://github.com/rajveer100704/semanticmemo/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/rajveer100704/semanticmemo/blob/main/docs/results.md
Author: Rajveer Singh Saggu
License-Expression: MIT
License-File: LICENSE
Keywords: agents,cross-encoder,embeddings,equivalence,faiss,false-positive-reduction,llm,mlp-classifier,risk-aware,semantic-cache,semantic-memory
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.15,>=3.11
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pydantic<3,>=2.7
Provides-Extra: dev
Requires-Dist: pyright<2,>=1.1.400; extra == 'dev'
Requires-Dist: pytest-asyncio<2,>=0.24; extra == 'dev'
Requires-Dist: pytest<10,>=8; extra == 'dev'
Requires-Dist: ruff<1,>=0.9; extra == 'dev'
Provides-Extra: kafka
Requires-Dist: aiokafka>=0.10.0; extra == 'kafka'
Provides-Extra: llm
Requires-Dist: anthropic<1,>=0.60; extra == 'llm'
Requires-Dist: openai<3,>=1.70; extra == 'llm'
Provides-Extra: ml
Requires-Dist: faiss-cpu<2,>=1.8; extra == 'ml'
Requires-Dist: sentence-transformers<6,>=3; extra == 'ml'
Requires-Dist: torch<3,>=2.3; extra == 'ml'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]<4,>=3.1; extra == 'postgres'
Provides-Extra: production
Requires-Dist: aiokafka>=0.10.0; extra == 'production'
Requires-Dist: opentelemetry-api>=1.20; extra == 'production'
Requires-Dist: opentelemetry-sdk>=1.20; extra == 'production'
Requires-Dist: prometheus-client>=0.20; extra == 'production'
Requires-Dist: psycopg[binary]<4,>=3.1; extra == 'production'
Requires-Dist: qdrant-client>=1.9.0; extra == 'production'
Requires-Dist: redis>=5.0.0; extra == 'production'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.9.0; extra == 'qdrant'
Provides-Extra: redis
Requires-Dist: redis>=5.0.0; extra == 'redis'
Provides-Extra: telemetry
Requires-Dist: opentelemetry-api>=1.20; extra == 'telemetry'
Requires-Dist: opentelemetry-sdk>=1.20; extra == 'telemetry'
Requires-Dist: prometheus-client>=0.20; extra == 'telemetry'
Description-Content-Type: text/markdown

# SemanticMemo

**Authorization-aware, resource-versioned distributed semantic caching infrastructure for LLM agents.**

[![Tests](https://github.com/rajveer100704/semanticmemo/actions/workflows/ci.yml/badge.svg)](https://github.com/rajveer100704/semanticmemo/actions)
[![PyPI](https://img.shields.io/pypi/v/semanticmemo.svg?v=2)](https://pypi.org/project/semanticmemo/)
[![Python](https://img.shields.io/pypi/pyversions/semanticmemo.svg?v=2)](https://pypi.org/project/semanticmemo/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

---

## The Evolution: V1 to V2

### Why Cosine Caching Fails in Production
Semantic caching is essential for scaling LLM agent workflows and keeping API costs predictable. However, naive cosine caches make a critical mistake: they treat embedding proximity as equivalence.
```
"Should I approve the refund?"   ── cosine: 0.97 ──▶  CACHE HIT  ← catastrophic false positive
"Should I deny the refund?"      ─────────────────────────────────────────
```
Opposite-action prompts sit dangerously close in embedding space. In finance, healthcare, or enterprise agents, serving a cached answer across opposite prompts creates silent, catastrophic bugs.

**SemanticMemo v1.x** solved this semantic failure by replacing the cosine threshold with a 4-stage verification pipeline (FAISS retrieval + lightweight MLP equivalence classifier + Cross-Encoder re-ranker + 11-rule regex entity drift veto).

### The Enterprise Gap Solved by V2.0.0
Even with perfect semantic equivalence, caching in real-world multi-tenant agent architectures introduces deep distributed systems and security challenges:
1. **Cross-Tenant Data Leakage:** A prompt from Tenant A must never reuse cached responses generated for Tenant B.
2. **Resource Permission Invalidation:** If User X loses access to Document 42, a cached answer must never grant access. Cache hits cannot bypass current authorization.
3. **Stale Resource States:** When Document 42 updates from version 7 to version 8, cached inferences tied to version 7 are logically invalid.
4. **Distributed Stampedes & Split-Brain Writes:** Under high concurrency across multiple container instances, hundreds of identical requests must coalesce into exactly one LLM call without stale leaders overwriting newer cache states.

**SemanticMemo 2.0.0** elevates the framework from a single-process library into a full **authorization-aware, resource-versioned distributed caching plane**.

---

## Retrieval Quality & Cache Safety Evaluation

SemanticMemo includes offline evaluation tooling for retrieval ranking and
high-stakes opposite-action prompts. The results below come from small,
manually curated closed-corpus fixtures, not production traffic.

### Retrieval relevance benchmark

The expanded fixture contains 42 queries (14 canonical queries and 28 paraphrases),
42 candidate documents, and 14 intent groups. Both methods use the same candidate
corpus and relevance labels.

| Metric | Sentence Transformer | TF-IDF baseline |
| :--- | ---: | ---: |
| Recall@5 | 97.62% | 70.63% |
| Precision@5 | 58.57% | 42.38% |
| MRR@10 | 95.63% | 80.10% |
| Top-1 intent accuracy | 92.86% | 66.67% |
| Cross-intent top-1 rate | 7.14% | 33.33% |

The Sentence Transformer uses `sentence-transformers/all-MiniLM-L6-v2`.
TF-IDF is a deterministic lexical reference. These results characterize this fixture
only; larger, independently labelled datasets and production workloads may differ.

### High-stakes opposite-action evaluation

A separate hand-authored evaluation contains 28 pairs: 12 equivalent pairs and
16 opposite-action pairs. It compares cosine similarity at threshold `0.90` with
the equivalence classifier at threshold `0.95`.

| Metric | Cosine baseline | Equivalence classifier |
| :--- | ---: | ---: |
| Precision | 33.3% | 60.0% |
| Recall | 33.3% | 75.0% |
| F1 | 33.3% | 66.7% |
| Opposite-action false positives | 8 / 16 | 6 / 16 |

On this fixture, the classifier blocked four dangerous candidate cache hits accepted
by the cosine baseline. This adversarial evaluation is not a safety certification
or a guarantee that every conflicting request will be detected.

Regression tests also cover conflicting actions and unrelated prompts that should
not be treated as opposites.

---

## Core Architecture & Execution Pipeline

In SemanticMemo 2.0.0, authorization and resource identity are evaluated **before** vector retrieval, and distributed single-flight guarantees strictly one LLM execution across all worker nodes.

```mermaid
flowchart TD
    Client["Caller: cache.get_or_call(...)"] --> Step1["① Authenticate Principal (user_id)"]
    Step1 --> Step2["② Derive Authoritative Tenant Context"]
    Step2 --> Step3["③ Evaluate Current RBAC & Permissions"]
    Step3 --> Step4["④ Resolve Fail-Closed RetrievalScope"]
    Step4 --> Step5["⑤ Scoped Vector Search (Qdrant / In-Memory)"]
    Step5 --> Step6{"⑥ Semantic Equivalence Pipeline\n(MLP + Cross-Encoder + Entity Veto)"}
    
    Step6 -->|"Valid Cache Hit"| Hit["✅ Return Authoritative Result"]
    Step6 -->|"Cache Miss"| Step7["⑦ Distributed Redis Single-Flight\n(Monotonic Fencing Generation)"]
    
    Step7 -->|"Leader Node"| CallLLM["Execute LLM Provider Callback"]
    Step7 -->|"Follower Nodes"| AwaitLeader["Await Completion / Poll Scoped Cache"]
    
    CallLLM --> CAS["Validate Fencing Token (CAS Guard)"]
    CAS --> StorePG["⑧ Commit to PostgreSQL & Invalidate Old Versions"]
    StorePG --> Outbox["⑨ Transactional Outbox Record"]
    Outbox --> Kafka["⑩ Kafka Event Relay (sm.events.resource_updated)"]
    Kafka --> Consumer["⑪ Idempotent Consumer & Qdrant Reconciliation"]
    StorePG --> Hit
    AwaitLeader --> Hit
```

### The 11-Stage Pipeline Explained:
1. **Authentication:** Caller identity (`user_id`) is strictly authenticated fail-closed.
2. **Authoritative Tenancy:** Tenant context is derived server-side from the authenticated user (`user.tenant_id`), completely preventing client tenant spoofing.
3. **Current RBAC:** Evaluates user role and permissions in PostgreSQL/SQLite prior to vector search.
4. **RetrievalScope:** Resolves the retrieval boundary into one of three distinct scopes:
   - **`GLOBAL`**: Shared, public data accessible system-wide.
   - **`TENANT`**: Strictly isolated to the caller's tenant (`scope=tenant AND tenant_id=current_tenant`).
   - **`RESOURCE`**: Bound to a specific `resource_id` and its current `resource_version`.
5. **Server-Side Filtered Retrieval:** Vector search pre-filters strictly within the resolved scope. Cross-tenant or stale-version vectors are excluded at the index layer.
6. **4-Stage Verification:** Evaluates candidate equivalence via MLP classifier, Cross-Encoder joint-attention pass, and regex entity drift veto.
7. **Distributed Single-Flight:** Redis distributed lock coordinates concurrent stampedes across multiple OS processes and nodes.
8. **Monotonic CAS Generation Fencing:** Protects cache writes against stalled leaders whose lock TTL expired. Stale generations are rejected by the database.
9. **Authoritative Relational Store:** PostgreSQL acts as the single source of truth for all entries, metadata, and versioning.
10. **Transactional Outbox & Kafka Relay:** State changes and outbox events are committed in a single atomic database transaction, then relayed durably to Kafka.
11. **Idempotent Invalidation & Reconciliation:** Background workers consume invalidation events deduplicated by an authoritative event ledger. The `IndexReconciler` automatically heals vector index drift against PostgreSQL.

---

## Installation & Packaging

SemanticMemo maintains an ultra-lightweight core with zero mandatory distributed drivers. Install only the capabilities required for your environment.

### Core Installation (Lightweight Local Dev)
```bash
pip install semanticmemo
```
*Dependencies: `numpy`, `pydantic`.*  
Operates fully with local SQLite, in-memory vector indexing, and local in-process single-flight.

### Production Integrations (Optional Extras)
```bash
# Neural ML Models (PyTorch, SentenceTransformers, FAISS)
pip install "semanticmemo[ml]"

# PostgreSQL authoritative storage & RBAC
pip install "semanticmemo[postgres]"

# Redis distributed single-flight & generation fencing
pip install "semanticmemo[redis]"

# Qdrant server-side scoped vector search
pip install "semanticmemo[qdrant]"

# Kafka durable outbox event relay
pip install "semanticmemo[kafka]"

# OpenTelemetry distributed tracing & Prometheus metrics
pip install "semanticmemo[telemetry]"

# Full Production Enterprise Suite (All integrations)
pip install "semanticmemo[production]"
```

---

## Quickstart

### 1. Breaking Change Notice (v2.0.0)

> [!IMPORTANT]
> **Authentication is now fail-closed.** In SemanticMemo 2.0.0, `get_or_call(...)` strictly requires an authenticated `user_id`. Unauthenticated invocations immediately raise `PermissionError`. For local development and test scripts, call `cache.bootstrap_default_user()` or pass `user_id="default_user"`.

### 2. Local Development (SQLite & In-Memory)

```python
import asyncio
from semanticmemo import SemanticMemo

async def call_llm(prompt: str) -> str:
    # Your LLM call (OpenAI, Anthropic, Ollama, etc.)
    return f"Response for: {prompt}"

async def main():
    # Initializes local SQLite cache store & in-memory single flight
    cache = SemanticMemo(domain="customer-support")
    
    # Bootstrap default principal for local dev
    cache.bootstrap_default_user(user_id="alice", tenant_id="dev_corp", role="analyst")

    result = await cache.get_or_call(
        prompt="How do I reset my password?",
        llm_function=call_llm,
        user_id="alice",
    )

    print(f"Result: {result.response}")
    print(f"Was Cache Hit: {result.was_cache_hit}")
    print(f"Latency: {result.latency_ms:.2f}ms")

if __name__ == "__main__":
    asyncio.run(main())
```

### 3. Distributed Production Configuration

Deploy with PostgreSQL, Redis, Qdrant, and OpenTelemetry:

```python
from semanticmemo import (
    SemanticMemo,
    CacheConfig,
    PostgresConfig,
    RedisConfig,
    QdrantConfig,
)

cache = SemanticMemo(
    domain="finance",
    postgres_config=PostgresConfig(
        dsn="postgresql://app_user:secret@postgres.internal:5432/semanticmemo",
        pool_size=20,
    ),
    redis_config=RedisConfig(
        url="redis://redis.internal:6379/0",
        lock_ttl_ms=5000,
        wait_timeout_ms=5000,
    ),
    qdrant_config=QdrantConfig(
        url="http://qdrant.internal:6333",
        collection_name="finance_cache_v2",
    ),
)

# Register a multi-tenant versioned resource
cache.register_resource(
    resource_id="quarterly_report_q3",
    tenant_id="hedge_fund_a",
    initial_version=1,
)

# Subsequent queries are scoped to hedge_fund_a and quarterly_report_q3 v1
result = await cache.get_or_call(
    prompt="Summarize the operating margin from the Q3 report",
    llm_function=call_llm,
    user_id="trader_bob",
    resource_id="quarterly_report_q3",
)
```

---

## Scopes & Multi-Tenant Versioning

SemanticMemo 2.0.0 enforces three distinct retrieval scopes:

| Scope | Visibility | Filter Condition | Typical Use Case |
| :--- | :--- | :--- | :--- |
| **`GLOBAL`** | All Tenants | `scope = 'global'` | Public knowledge bases, general product FAQs, static system prompts |
| **`TENANT`** | Single Tenant | `scope = 'tenant' AND tenant_id = current_tenant` | Customer support tickets, private organization data |
| **`RESOURCE`** | Single Resource & Version | `resource_id = id AND resource_version = N` | Deal rooms, sensitive contracts, uploaded documents |

### Dynamic Cache Identity
Cache entries in SemanticMemo 2.0.0 are uniquely keyed across six dimensions:
```
flight_key = sm:lock:{pipeline_version}:{scope}:{tenant_id}:{resource_id}:{resource_version}:{prompt_sha256}
```
When a document increments from `resource_version = 1` to `resource_version = 2`, older cache entries are immediately bypassed without requiring synchronous vector deletions. PostgreSQL remains the correctness authority.

---

## Observability & Security Sanitization

SemanticMemo 2.0.0 ships with built-in OpenTelemetry tracing and Prometheus metrics designed for strict enterprise security compliance:
- **Zero PII in Attributes:** Trace attributes and metric labels automatically hash or redact sensitive fields (API keys, raw prompts, passwords, user identifiers).
- **OpenMetrics Export:** `cache.metrics.export_openmetrics_text()` provides Prometheus-compatible gauges, counters, and latency histograms.
- **Explainable Decision Traces:** Every result includes stage-by-stage latencies (`embedding_latency_ms`, `retrieval_latency_ms`, `mlp_latency_ms`, `cross_encoder_latency_ms`).

---

## Migration from SQLite to PostgreSQL

For teams transitioning from SemanticMemo v1.x SQLite stores to PostgreSQL, SemanticMemo 2.0.0 provides a migration utility with deterministic SHA-256 row checksum validation:

```python
from semanticmemo.store.migration import migrate_sqlite_to_postgres
from semanticmemo.store import SQLiteCacheStore, PostgresCacheStore, PostgresConfig

sqlite_store = SQLiteCacheStore(".semanticmemo/cache.db")
postgres_store = PostgresCacheStore(PostgresConfig(dsn="postgresql://..."))

report = migrate_sqlite_to_postgres(sqlite_store, postgres_store)
print(f"Migration Complete: {report.is_valid}")
print(f"Total Rows Migrated: {report.total_migrated}")
```

---

## Release Validation & Certification

SemanticMemo 2.0.0 distinguishes between standard GitHub Actions CI and strict live infrastructure certification:

- **GitHub Actions CI:** Checks Python 3.11 through 3.14 with Ruff formatting and lint, Pyright type checking, and automated tests. In release preparation, the local full suite passed 253 tests with 14 skipped; environment-dependent tests may skip when external services are unavailable.
- **Certified Live Environment:** Validated against real live PostgreSQL 16, Redis 7, Qdrant 1.13.4, and Kafka 3.7.0 instances on Windows using 3 distinct OS processes:
  - **9/9 Core Live Certification Phases:** PASSED
  - **Lightweight Worker Import Safety Test:** PASSED (Asserts zero loading of `torch`, `scipy`, `sklearn`, `sentence_transformers`)
  - **Live Skips / Failures:** 0 skipped, 0 failed.
  - **Distributed Single-Flight Coalescing:** Verified (100 requests $\to$ exactly 1 LLM execution across 3 OS processes).

---

## License

MIT — see [LICENSE](LICENSE).
