Metadata-Version: 2.4
Name: scikit-ai
Version: 0.2.0
Summary: Task-oriented components for text, images, audio and video with a scikit-learn API, powered by open-source models or cloud APIs.
Author-Email: Georgios Douzas <gdouzas@icloud.com>
License-Expression: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Documentation
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Project-URL: Homepage, https://georgedouzas.github.io/scikit-ai
Project-URL: Documentation, https://georgedouzas.github.io/scikit-ai
Project-URL: Changelog, https://georgedouzas.github.io/scikit-ai/changelog
Project-URL: Repository, https://github.com/georgedouzas/scikit-ai
Project-URL: Issues, https://github.com/georgedouzas/scikit-ai/issues
Project-URL: Discussions, https://github.com/georgedouzas/scikit-ai/discussions
Project-URL: Funding, https://github.com/sponsors/georgedouzas
Requires-Python: <3.14,>=3.11
Requires-Dist: scikit-learn>=1.6.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: pillow>=10.4.0
Requires-Dist: soundfile>=0.12.1
Requires-Dist: httpx>=0.27.0
Provides-Extra: openai
Requires-Dist: openai>=1.82.0; extra == "openai"
Provides-Extra: hf
Requires-Dist: torch>=2.4.0; extra == "hf"
Requires-Dist: transformers>=4.46.0; extra == "hf"
Requires-Dist: sentence-transformers>=3.2.0; extra == "hf"
Requires-Dist: diffusers>=0.31.0; extra == "hf"
Requires-Dist: accelerate>=1.0.0; extra == "hf"
Provides-Extra: litellm
Requires-Dist: litellm>=1.50.0; extra == "litellm"
Provides-Extra: replicate
Requires-Dist: replicate>=1.0.0; extra == "replicate"
Provides-Extra: video
Requires-Dist: av>=13.0.0; extra == "video"
Provides-Extra: all
Requires-Dist: scikit-ai[hf,litellm,openai,replicate,video]; extra == "all"
Description-Content-Type: text/markdown

[black badge]: <https://img.shields.io/badge/%20style-black-000000.svg>
[black]: <https://github.com/psf/black>
[docformatter badge]: <https://img.shields.io/badge/%20formatter-docformatter-fedcba.svg>
[docformatter]: <https://github.com/PyCQA/docformatter>
[ruff badge]: <https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json>
[ruff]: <https://github.com/charliermarsh/ruff>
[mypy badge]: <http://www.mypy-lang.org/static/mypy_badge.svg>
[mypy]: <http://mypy-lang.org>
[properdocs badge]: <https://img.shields.io/badge/docs-properdocs-blue.svg?style=flat>
[properdocs]: <https://properdocs.org>
[pip-audit badge]: <https://img.shields.io/badge/security-pip--audit-green>
[pip-audit]: <https://github.com/pypa/pip-audit>
[bandit badge]: <https://img.shields.io/badge/security-bandit-yellow>
[bandit]: <https://github.com/PyCQA/bandit>
[pytest badge]: <https://img.shields.io/badge/tests-pytest-blue>
[pytest]: <https://github.com/pytest-dev/pytest>
[coverage badge]: <https://img.shields.io/badge/coverage-pytest--cov-blue>
[coverage]: <https://github.com/nedbat/coveragepy>
[interrogate badge]: <https://img.shields.io/badge/docstring-interrogate-blue>
[interrogate]: <https://github.com/econchick/interrogate>
[pre-commit badge]: <https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit>
[pre-commit]: <https://github.com/pre-commit/pre-commit>
[nox badge]: <https://img.shields.io/badge/%F0%9F%A6%8A-Nox-D85E00.svg>
[nox]: <https://github.com/wntrblm/nox>
[version badge]: <https://img.shields.io/pypi/v/scikit-ai.svg>
[pythonversion badge]: <https://img.shields.io/pypi/pyversions/scikit-ai.svg>
[downloads badge]: <https://img.shields.io/pypi/dd/scikit-ai>
[discussions]: <https://github.com/georgedouzas/scikit-ai/discussions>
[discussions badge]: <https://img.shields.io/github/discussions/georgedouzas/scikit-ai>
[ci]: <https://github.com/georgedouzas/scikit-ai/actions?query=workflow>
[ci badge]: <https://github.com/georgedouzas/scikit-ai/actions/workflows/ci.yml/badge.svg?branch=main>
[doc]: <https://github.com/georgedouzas/scikit-ai/actions?query=workflow>
[doc badge]: <https://github.com/georgedouzas/scikit-ai/actions/workflows/doc.yml/badge.svg?branch=main>

# scikit-ai

[![ci][ci badge]][ci] [![doc][doc badge]][doc]

| Category          | Tools    |
| ------------------| -------- |
| **Development**   | [![black][black badge]][black] [![ruff][ruff badge]][ruff] [![mypy][mypy badge]][mypy] [![docformatter][docformatter badge]][docformatter] |
| **Testing**       | [![pytest][pytest badge]][pytest] [![coverage][coverage badge]][coverage] [![interrogate][interrogate badge]][interrogate] |
| **Security**      | [![pip-audit][pip-audit badge]][pip-audit] [![bandit][bandit badge]][bandit] |
| **Automation**    | [![nox][nox badge]][nox] [![pre-commit][pre-commit badge]][pre-commit] |
| **Package**       | ![version][version badge] ![pythonversion][pythonversion badge] ![downloads][downloads badge] |
| **Documentation** | [![properdocs][properdocs badge]][properdocs]|
| **Communication** | [![discussions][discussions badge]][discussions] |

## Introduction

scikit-ai provides task-oriented components for text, images, audio and video. Each component does one job, such as
describing an image, summarizing a text, transcribing speech or making a portrait speak, and follows the
[scikit-learn](https://scikit-learn.org/stable) estimator API. The same component runs on open-source models or on cloud
APIs, chosen with one `backend` parameter, and fits into `Pipeline`, `GridSearchCV` and the rest of the scikit-learn
ecosystem.

## Installation

scikit-ai is available on PyPI. The base install includes the offline `dummy` backend, and every other backend is an
optional extra:

```bash
pip install scikit-ai              # components and the dummy backend
pip install "scikit-ai[hf]"        # local open-source models with Hugging Face
pip install "scikit-ai[openai]"    # the OpenAI API
pip install "scikit-ai[litellm]"   # Anthropic, Gemini, Mistral, Bedrock, Ollama and others through LiteLLM
pip install "scikit-ai[replicate]" # open-source models hosted by Replicate
pip install "scikit-ai[video]"     # encoding and decoding video
pip install "scikit-ai[all]"       # everything
```

Development installation requires to clone the repository and then use [PDM](https://github.com/pdm-project/pdm) to install the
project as well as the main and development dependencies:

```bash
git clone https://github.com/georgedouzas/scikit-ai.git
cd scikit-ai
pdm install
```

## Components

Components live in modules named after their task family, like scikit-learn's estimators. Every class name starts
with its input type, except generators, which are named for what they generate.

| Module | Component | Does | Output |
|---|---|---|---|
| `skai.classification` | `TextClassifier`, `ImageClassifier`, `AudioClassifier`, `VideoClassifier` | assign labels, zero-shot or few-shot | labels and probabilities |
| `skai.description` | `ImageDescriptor`, `AudioDescriptor`, `VideoDescriptor` | describe media in natural language | text |
| `skai.transcription` | `SpeechTranscriber`, `VideoTranscriber` | transcribe speech, optionally with timestamps | text or timed records |
| `skai.text` | `TextSummarizer` | summarizes texts | text |
| `skai.text` | `TextTranslator` | translates texts | text |
| `skai.text` | `TextExtractor` | extracts named fields | records or columns |
| `skai.embedding` | `TextEmbedder`, `ImageEmbedder`, `AudioEmbedder` | map samples to vectors | float array |
| `skai.generation` | `SpeechSynthesizer` | speaks texts, optionally in a cloned voice | audio |
| `skai.generation` | `ImageGenerator`, `VideoGenerator` | generate images or videos from prompts | images or videos |
| `skai.generation` | `ImageEditor` | applies an instruction to images | images |
| `skai.generation` | `ImageAnimator` | turns images into short videos | videos |
| `skai.generation` | `TalkingAvatarGenerator` | animates a portrait so that it speaks the audio | video |

## What fit learns

Every component learns from the data given to `fit`, without training a model unless you ask for fine-tuning.

<!-- --8<-- [start:fit] -->
| Component | `fit(X, y)` takes | Learns | Uses it as |
|---|---|---|---|
| `TextClassifier`, `ImageClassifier` | samples and labels | `classes_`, `examples_` | one example per class by default |
| `TextSummarizer` | texts and reference summaries | `examples_`, `max_length_` | example pairs and a learned length |
| `TextTranslator` | texts and reference translations | `examples_`, `glossary_` | example pairs and consistent terms |
| `ImageDescriptor` | images and reference descriptions | `examples_` | example pairs in the reference style |
| `SpeechTranscriber` | audio and reference transcripts | `vocabulary_` | domain terms sent as spelling context |
| `TextEmbedder` | texts | `mean_`, `components_` | centering, and PCA when the backend cannot shorten |
| `SpeechSynthesizer` | the `voice` parameter | `voice_` | a voice cloned from one or more recordings |
| `ImageGenerator` | the `style_images` parameter | `style_` | a style learned from reference images |
| `TalkingAvatarGenerator` | the `avatar` parameter | `avatar_` | the portrait cropped and sized for the backend |

With `adaptation='finetune'`, classifiers, `TextSummarizer`, `TextTranslator` and `ImageDescriptor` train a model on
all the training data on backends that support fine-tuning, and `fit` raises `UnsupportedBackendError` on the others.
In a pipeline, intermediate steps are fitted through `fit_transform`, which ignores `y`, so a transcriber in front of a
classifier never reads the class labels as transcripts.
<!-- --8<-- [end:fit] -->

## Usage

Every component is a scikit-learn estimator. The examples use the offline `dummy` backend, so they run without models
or credentials. Replace `backend='dummy'` with a backend from the table below to get real results.

Classify texts with a few labeled examples:

```python
from skai.classification import TextClassifier

X = ['I loved this movie.', 'The product broke after a day.', 'Great service!', 'Never again.']
y = ['positive', 'negative', 'positive', 'negative']

classifier = TextClassifier(k_shot=2, random_state=0, backend='dummy').fit(X, y)
classifier.predict(['What a wonderful day.'])
```

Describe images, then classify the descriptions in one pipeline:

```python
from PIL import Image
from sklearn.pipeline import make_pipeline
from skai.description import ImageDescriptor

images = [Image.new('RGB', (64, 64), color) for color in ('red', 'green', 'blue', 'white')]
pipeline = make_pipeline(ImageDescriptor(backend='dummy'), TextClassifier(backend='dummy'))
pipeline.fit(images, ['warm', 'cold', 'cold', 'warm']).predict(images)
```

Make a portrait speak a text:

```python
from skai.generation import TalkingAvatarGenerator
from skai.generation import SpeechSynthesizer

portrait = Image.new('RGB', (64, 64), 'white')
pipeline = make_pipeline(
    SpeechSynthesizer(backend='dummy'),
    TalkingAvatarGenerator(avatar=portrait, backend='dummy'),
)
videos = pipeline.fit(None).predict(['Hello, I am a talking portrait.'])
videos[0].duration
```

## Backends

The `backend` parameter takes `'<provider>'`, `'<provider>:<model>'` or a backend object from `skai.backends`. Without
it, components run an open-source model.

<!-- --8<-- [start:providers] -->
| Provider | Backend | Runs | Install | Tasks |
|---|---|---|---|---|
| `hf` | `HFBackend` | locally, open-source models | `scikit-ai[hf]` | every task except talking avatars |
| `openai` | `OpenAIBackend` | OpenAI API | `scikit-ai[openai]` | every task except talking avatars and image and audio embeddings |
| `litellm` | `LiteLLMBackend` | Anthropic, Gemini, Mistral, Bedrock, Azure, Ollama and others through LiteLLM | `scikit-ai[litellm]` | classification, description, summarization, translation, extraction, text and image embeddings |
| `replicate` | `ReplicateBackend` | open-source models hosted by Replicate | `scikit-ai[replicate]` | talking avatars, image generation, speech synthesis with voice cloning |
| `dummy` | `DummyBackend` | locally, no model | included | every task, with placeholder outputs for tests and documentation |
<!-- --8<-- [end:providers] -->

The default model of every component and provider:

<!-- --8<-- [start:defaults] -->
| Component | Default backend | `hf` | `openai` | `litellm` | `replicate` |
|---|---|---|---|---|---|
| `TextClassifier` | `hf` | `facebook/bart-large-mnli` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `TextSummarizer` | `hf` | `Qwen/Qwen2.5-1.5B-Instruct` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `TextTranslator` | `hf` | `Qwen/Qwen2.5-1.5B-Instruct` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `TextEmbedder` | `hf` | `sentence-transformers/all-MiniLM-L6-v2` | `text-embedding-3-small` | `openai/text-embedding-3-small` | |
| `SpeechSynthesizer` | `hf` | `facebook/mms-tts-eng` | `gpt-4o-mini-tts` | | `jaaari/kokoro-82m`, `lucataco/xtts-v2` to clone a voice |
| `ImageGenerator` | `hf` | `stabilityai/sd-turbo` | `gpt-image-1` | | `black-forest-labs/flux-schnell` |
| `ImageDescriptor` | `hf` | `HuggingFaceTB/SmolVLM-256M-Instruct` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `ImageClassifier` | `hf` | `openai/clip-vit-base-patch32` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `SpeechTranscriber` | `hf` | `openai/whisper-small` | `whisper-1` | | |
| `TalkingAvatarGenerator` | `replicate` | | | | `cjwbw/sadtalker` |
| `AudioClassifier` | `hf` | `laion/clap-htsat-unfused` | `gpt-audio-mini` | `gemini/gemini-2.5-flash` | |
| `VideoClassifier` | `hf` | `HuggingFaceTB/SmolVLM-256M-Instruct` | `gpt-5-mini` | `gemini/gemini-2.5-flash` | |
| `AudioDescriptor` | `hf` | `Qwen/Qwen2-Audio-7B-Instruct` | `gpt-audio-mini` | `gemini/gemini-2.5-flash` | |
| `VideoDescriptor` | `hf` | `HuggingFaceTB/SmolVLM-256M-Instruct` | `gpt-5-mini` | `gemini/gemini-2.5-flash` | |
| `VideoTranscriber` | `hf` | `openai/whisper-small` | `whisper-1` | | |
| `TextExtractor` | `hf` | `Qwen/Qwen2.5-1.5B-Instruct` | `gpt-5-mini` | `anthropic/claude-haiku-5-5` | |
| `ImageEmbedder` | `hf` | `clip-ViT-B-32` | | `cohere/embed-v4.0` | |
| `AudioEmbedder` | `hf` | `laion/clap-htsat-unfused` | | | |
| `VideoGenerator` | `hf` | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | `sora-2` | | |
| `ImageEditor` | `hf` | `timbrooks/instruct-pix2pix` | `gpt-image-1` | | |
| `ImageAnimator` | `hf` | `stabilityai/stable-video-diffusion-img2vid-xt` | `sora-2` | | |
<!-- --8<-- [end:defaults] -->

Cloud backends read credentials from environment variables, such as `OPENAI_API_KEY` or `REPLICATE_API_TOKEN`, and
never take a secret as an argument. Generated speech and talking avatars can imitate real people. Use them only with
the consent of the people they depict, and within the policies of the providers.

## Spec-driven development

The project's engineering constitution lives in `openspec/constitution.md` and binds all work. The project uses
[OpenSpec](https://github.com/Fission-AI/OpenSpec) for spec-driven development, and `openspec/config.yaml` names the
constitution as binding.
