Skip to content

Desktop App (Tauri)

The desktop UI is built with Tauri 2 + React 18 + TypeScript + Vite 6 + Zustand + Tailwind CSS v4. It communicates with the Python backend over HTTP + SSE on 127.0.0.1:8765.

Layout

The frontend is structured as a multi-panel desktop application:

App.tsx
├── TitleBar       # Custom window chrome (32px height)
├── Content (flex:1)
│   ├── ChatWindow  # Message list + auto-scroll + streaming animation
│   └── Sidebar     # 320px collapsible, 11 panel sections
└── StatusBar       # Connection state, voice state (28px height)

ChatWindow

The primary interaction surface. Renders messages from the SSE stream with: - Auto-scroll to latest message on new content - Streaming animation for in-progress agent responses - Tool call visualization (expanded/collapsed) - Empty state guidance when no messages exist - ChatInput at the bottom with send button and keyboard shortcuts - Message grouping heuristic — groupMessages() combines consecutive same-speaker assistant messages into a single rendered bubble on reload (matches live-streaming appendToLast behavior). Tool and user messages remain separate.

The inner wrapper constrains width to maxWidth: 900px for readability.

Panel Description
ParamsPanel All sampling params stored in Zustand store (not local state). Includes message_slice_len slider (range 3-100) for controlling agent context window
ModesPanel 10 mode toggles with optimistic updates
AgentsPanel Active agent selection and configuration
ModelsPanel Model dropdown for chat/embedding models
PromptsPanel System prompt preview and customization
MemoryManagementPanel Vector memory management
ToolPermissionsPanel Tool permission configuration
GrantsPanel Persistent tool approval grants display
BackupPanel Conversation backup and restore
LegacyMigrationPanel Legacy data migration tools
ActionsPanel Export conversations, clear chat, debug actions

Communication Protocol

Backend → Frontend: SSE Events

Agent responses stream via Server-Sent Events: 

event: ui_message
data: {"speaker": "agent", "content": "Hello!"}

event: tool_call
data: {"tool": "web_search", "args": {"query": "..."}}

The useEvents() hook maintains an EventSource connection, parses SSE messages, and routes them to Zustand stores. Key event types: - PARAM_CHANGED → calls setParam() in uiStore - CONFIG_UPDATED (type="model") → syncs model dropdown - SYSTEM_COMMAND → triggers system-level UI actions

Frontend → Backend: HTTP Commands

Commands are sent via POST /commands: 

{"command": "send_message", "payload": {"text": "Hello"}}
{"command": "set_param", "payload": {"key": "temperature", "value": 0.8}}
{"command": "set_mode", "payload": {"mode": "memory_mode", "value": true}}

The useCommands() hook provides typed wrappers: sendMessage(), setParam(), setMode().

Authentication

The API server requires an X-Session-Secret header with constant-time comparison. CORS middleware (allow_origins=["*"]) is safe because the server only binds to 127.0.0.1.

Zustand Stores

Store Purpose
chatStore Messages, streaming state, event handlers for UI events
uiStore Sidebar state, theme, ALL params (Record<string, number \| string>), modes, agents
grantsStore Persistent tool approval grants display

ParamsPanel Architecture

ALL params are stored in uiStore.params — never use local React state for persisted values. Read from storeParams['key']. Send changes via setParam(). This ensures values persist across panel collapse/expand and stay synced with the backend.

Chat Log Protocol

Messages written to ui/chat.log follow a delimiter protocol: - [[START:<id>:<speaker>]] ... [[END:<id>]] — agent messages - Lines without [[START]] — user messages - Tool calls and control directives are also logged

Text Message Flow

When the user types a message in the Tauri frontend:

  1. sendMessage() dispatches HTTP POST to /commands
  2. Backend sets stop_recording_event — halts active recording
  3. Backend sets text_only_flag — skips audio transcription
  4. Typed text enters agent pipeline directly
  5. Response streams back via SSE to chatStore

Think Mode Synchronization

Think mode requires multi-layer sync: 1. ParamsPanel reads from Zustand store (not local state) 2. useEvents routes setParam SSE events → uiStore update 3. Backend directly sets agent.think on all agents via _apply_control_update("think_mode") 4. uiStore type widened to accept think mode string values

See Also