Vector Companion¶

A multi-modal AI assistant that runs entirely on your machine with local LLM inference via Ollama, persistent vector memory via ChromaDB, real-time audio processing, and a Tauri 2 React desktop frontend.

At a Glance

Category	Detail
Language	Python 3.12
Inference	Multi-provider: Ollama (default), llama.cpp (OpenAI-compatible)
Memory	ChromaDB vector store + SQLite canonical persistence
Audio	Parakeet-TDT STT + Chatterbox TTS (faster fork)
Desktop UI	Tauri 2 + React 18 + TypeScript
Platform	Windows-first, cross-compatible
License	MIT

Key Features¶

graph LR
    A[Voice Input] --> B[STT Transcription]
    B --> C[Agent Generation]
    C --> D[Tool Calling]
    D --> E[TTS Synthesis]
    E --> F[Audio Output]
    C --> G[Tauri Desktop UI]
    C --> H[Vector Memory]

Modular Agent System¶

Agents are config data, not code. Add, remove, or replace agents by editing config/config.py — no Python changes required. The factory pattern (build_agents()) creates agents lazily at runtime.

Voice-First Interaction¶

Record speech → transcribe via Parakeet-TDT → stream agent response → synthesize with Chatterbox TTS → play audio. The entire pipeline runs locally with sub-second latency.

Ten Toggleable Modes¶

Switch between analysis, memory, auto-chat, control, cloud, and seven more modes — via voice command or UI toggle. Each mode changes agent behavior, model selection, or data flow.

Secure Tool Execution¶

Eleven tools route through an 8-step security pipeline (RealToolHost): manifest validation, autonomy policy checks, persistent grants/denials, human-in-the-loop approval, process isolation, timeout enforcement, and audit logging.

Background Research¶

Delegate long-running work to async workers. The web research protocol performs DuckDuckGo searches, extracts full page content via Chrome TLS impersonation, and synthesizes findings — all in the background.

Process Isolation¶

Risky tool executions (file writes, destructive operations) are routed through sandboxed subprocesses with path/domain validation and Windows job object containment.

Getting Started¶

# Clone and set up
git clone https://github.com/SingularityMan/vector_companion2.git
cd vector_companion2
uv sync

# Install TTS engine
git clone -b faster https://github.com/rsxdalv/chatterbox.git chatterbox
cd chatterbox && uv pip install -e . && cd ..

# Start the assistant
python main.py

Quickstart

Installation, prerequisites, first run. Get up and running in minutes.

Quickstart
Architecture

Three concurrent flows, 17 subpackages, data flow diagrams.

Architecture
Guides

Deep dives into agents, security, audio, data, the desktop app, and more.

Guides
API Reference

Auto-generated documentation from docstrings for every subpackage.

API Reference