Skip to content

Quickstart

Get Vector Companion running in three steps: install dependencies, pull Ollama models, and start the app.

Prerequisites

Requirement Version Purpose
Python 3.12+ Runtime
uv Latest Package manager
git Any Version control
Ollama Latest Local LLM inference
conda or Miniconda Latest pynini dependency for TTS
Node.js 18+ Tauri frontend (optional)
Rust toolchain Stable Tauri builds (optional)
Optional but Recommended
  • GPU: NVIDIA GPU with CUDA 12.8+ for faster inference and audio processing
  • RAM: 16 GB minimum, 32 GB recommended (models + ChromaDB + browser)

Step 1: Clone and Install

# Clone the repository
git clone https://github.com/SingularityMan/vector_companion2.git
cd vector_companion2

# Install Python dependencies
uv sync

# Install Chatterbox TTS engine (faster fork)
git clone -b faster https://github.com/rsxdalv/chatterbox.git chatterbox
cd chatterbox && uv pip install -e . && cd ..

# Install PyTorch with CUDA 12.8 (skip if CPU-only)
uv pip install torch==2.7.1+cu128 torchaudio==2.7.1+cu128 torchvision==0.22.1+cu128 \
    --extra-index-url https://download.pytorch.org/whl/cu128

# Install pynini (required by TTS engine)
conda install -c conda-forge pynini

Platform Notes

On Windows, activate the virtual environment before running:

.venv\Scripts\activate

On macOS/Linux:

source .venv/bin/activate

Step 2: Pull Ollama Models

Vector Companion requires specific Ollama models. Pull them from the terminal:

# Language model (main inference)
ollama pull qwen3:8b

# Embedding model (vector memory)
ollama pull qwen3-embedding:0.6b

# Vision model (screenshot captioning — optional)
ollama pull llava

Model Selection

The exact model names are configured in config/config.py. If you use different models, update the config values (language_model, vision_model1, etc.) to match what you've pulled.

Optional: llama.cpp (alternative provider)

Vector Companion supports llama.cpp as an alternative to Ollama via the ModelBackend Protocol. Install system-wide using your platform's package manager:

Windows — conda-forge (recommended, includes CUDA): 

conda install -c conda-forge llama.cpp

macOS / Linux — Homebrew: 

brew install llama.cpp

Verify the installation: llama --help. To use llama.cpp, start llama-server with your model and set config.llm_provider = "llama_cpp". See the Multi-Provider LLM guide for details.

Step 3: Start the App

Backend (Required)

python main.py

The assistant begins listening for voice input immediately. The local API server starts on 127.0.0.1:8765.

Tauri Frontend (Optional)

Open a second terminal:

cd tauri/frontend
npm install    # First time only
npm run dev    # Opens Vite dev server at http://localhost:1420

The desktop app connects to the running backend automatically.

Verification Checklist

Confirm everything is working:

  • [ ] python main.py starts without import errors
  • [ ] Console shows "API server started on 127.0.0.1:8765"
  • [ ] Ollama responds: curl http://127.0.0.1:11434/api/tags lists your models
  • [ ] Tauri frontend (if running) connects to the backend (check browser console)
  • [ ] Speak a sentence — you should see transcription in the console

Environment Variables

Optional configuration via environment variables:

Variable Default Description
VECTOR_DB_PATH ./data/chromadb ChromaDB persistent storage path
VECTOR_RERANKER_DIRS . Semicolon-separated directories for reranker model
VECTOR_MIC_INDEX 0 Microphone device index
OLLAMA_HOST http://127.0.0.1:11434 Ollama API endpoint
Using a custom microphone

List available devices with Python:

import sounddevice as sd
for i, dev in enumerate(sd.query_devices()):
    print(f"{i}: {dev['name']} (in={dev['max_input_channels']}, out={dev['max_output_channels']})")

Then set VECTOR_MIC_INDEX=2 before running python main.py.

Next Steps