Vibe boldly. Verify everything.
Integrate AI coding agents into your scientific research.
Vaibify empowers you to develop, run, verify, publish, and reproduce AI-assisted data analysis. Work with coding agents in secure Docker containers, visually inspect figures and data, verify your pipeline steps were completed in order, and enable others to reproduce your results at the bit level.
The dashboard shows the up-to-date status of a research project in the secured computing environment. The selected step has passing dependencies but is pending researcher approval. Work with agents in the terminal and view outputs in the viewing windows.
- Left Workflow & verification
- Inspect each pipeline step’s commands, files, dependencies, and unit tests. When you are satisfied, approve it.
- Upper right Figures & data
- Examine outputs side by side and compare figures with saved standards.
- Lower right Terminals & agents
- Run commands, make interactive decisions, and work with an AI coding agent.
Motivation and Goals
Keep your computer safe and verify the AI’s work
Learn more about developing code with AIBecause AI agents are more capable software engineers than humans, researchers no longer write or inspect source code. The new bottleneck is verification. vaibify brings the agents, the figures, the data analysis pipeline, and the remote repositories into one view, significantly reducing the time needed to verify AI-generated code.
Docker containers keep AI agents contained in a locked-down computing environment, which has a specific set of dependencies that facilitates reproduction. From the host machine's terminal, you can access the container, but Docker containers restrict agents’ access to the host machine, allowing longer agent runs with less direct supervision.
Establishing an analysis's veracity still requires inspecting the data, testing the calculations, and understanding the assumptions. AI agents add a new level of uncertainty because they can behave stochastically. Continual monitoring of files in the container alerts the researcher to changes.
Since reproducing a computation does not establish its scientific accuracy, each step must be approved by a researcher for the entire workflow to be declared complete.
Researching with Vaibify
Safe exploration to reproducible workflow
Start in a sandbox, develop a software toolkit, and organize an investigation into a Project with explicit steps and dependencies.
Learn more about the Dashboard- Run coding agents
- Use Claude Code, Codex, or Gemini in the dashboard’s terminals. Docker provides a secure environment for the project’s code, dependencies, and agent sessions.
- Define the analysis
- Combine automatic and interactive steps. Each step records its input data, analysis commands, output files, plot commands, and dependencies on other steps.
- Inspect and test outputs
- View figures and text files alongside the terminal. Compare plots with saved standards and run integrity, qualitative, and quantitative tests. Researcher approval is recorded separately.
- Track what changed
- vaibify monitors the workflow’s files and declared dependencies. When an input or script changes, the dashboard identifies stale results and affected downstream steps that need attention.
- Convene an Agent Council
- Task multiple agents with generating independent plans for difficult tasks, empower them to work for multiple rounds, and chat with the chairbot when questions arise.
- Check published copies
- Compare local files with their GitHub and Zenodo copies using SHA-256 hashes. Optional Overleaf and arXiv checks help track whether manuscript figures match the local results.
- Enable bit-level reproducibility
- vaibify guides you through the steps to permanently store your data and computing environment for exact reproducibility. It also provides guidance on how to reproduce other vaibify archives at the bit level.
Reproducibility
The PROOF ladder
Provenance, Reproducibility, Openness, Oversight, Falsifiability. vaibify enables quantitative analyses to reach PROOF Level 3 out of 6.
Learn more about the PROOF Ladder| Level | Evidence |
|---|---|
| 1 Self-consistent | Declared inputs, current outputs, passing tests, researcher approval, and satisfied dependencies. |
| 2 Published | Matching copies on GitHub and Zenodo, together with a declaration of AI usage. |
| 3 Reproducible | A pinned container environment, a manifest of the project’s artifacts, and a recorded rerun whose output hashes match the manifest. |
| 4 Traceable | The public presentation of all system prompts, prompt transcripts, and a public changelog of all modifications to the digital artifacts from Level 1 to Level 3. |
| 5 Regenerated | An independent Level 4 reproduction of the results to within scientific tolerances by a different set of agents and system prompts. |
| 6 Attested | Multiple public and cryptographically verifiable statements by trusted entities that they achieved Level 5. |
Get started
Reproduce an existing project
Python 3.9 or later, on macOS or Linux.
pip install vaibify
vaibifyThe application opens locally in your browser and guides you through container and project creation. Follow the Quickstart guide to reproduce the AI Greenhouse example.