Insight

An AI research assistant for journalists. Your documents stay on your computer.

Why Insight?

Newsrooms deal with mountains of documents—court filings, FOIA responses, leaked files—but searching and analyzing them usually means uploading to cloud services you may not trust.

Insight is different. Your documents are processed, stored, and searched locally—they never leave your computer unless you choose to share them with a colleague. Chat runs on a cloud provider of your choice (OpenAI, Anthropic, Gemini, or a local server like Ollama), configured with your own API key.

What Can It Do?

• Answer questions about your documents — Ask in plain language, get answers with citations pointing to the exact source.
• Search smartly — Full-text search over your collection, driven by an AI agent that knows what to look for.
• Share securely — Send collections directly to colleagues, laptop-to-laptop, without a server in between.
• Own your documents — Storage, indexing, and sync all run locally; no account required.

Quick Start

1. Download Insight for your computer
2. Create a collection and add your documents
3. Start asking questions

See Getting Started for step-by-step instructions.

Getting Started

Download

Which file do I download?

• macOS: Download the .dmg file. Choose aarch64 for newer Macs (M1/M2/M3/M4) or x64 for older Intel Macs.
• Windows: Download the .exe installer. (IT departments may prefer the .msi.)
• Linux: Download .deb for Ubuntu/Debian, .rpm for Fedora/Red Hat, or .AppImage to run on any distribution.

Configure an AI Provider

Insight supports 25+ chat providers (OpenAI, Anthropic, Gemini, Groq, and local servers like Ollama and llama.cpp). Pick one in Settings and add an API key where required to get started. This works well on any computer.

Your documents are always processed and stored locally—only your questions and the AI’s answers go through the provider.

You don’t need the biggest model. Smaller models like GPT-5 mini or Claude Haiku work well with Insight’s agent—they’re faster, cheaper, and handle document research tasks effectively. The agent harness does the heavy lifting of breaking down queries and gathering evidence, so even lightweight models produce good results.

Create a Collection

Collections are folders for organizing your documents. You might create one for each investigation or story you’re working on.

1. Click New Collection
2. Give it a name (e.g., “City Budget Investigation”)
3. Drag and drop PDF files into the collection

Insight will process each document so the AI can search and read them. Note: text extraction works on digital PDFs; scanned PDFs (images of text) won’t be text-searchable.

Ask Questions

Once your documents are processed, just ask questions in plain language:

• “What was the total spending on consultants?”
• “Find all mentions of Company X”
• “Summarize the main findings from these reports”

The AI will search through your documents, find relevant passages, and give you an answer with citations you can click to see the original source.

Collections

Collections are how Insight organizes your documents. Think of them as folders—each collection holds a set of related documents with its own search and sharing settings.

Creating a Collection

Click New Collection and give it a name. You might organize by:

• Investigation (e.g., “City Hall Contracts”)
• Beat (e.g., “Climate”, “Housing”)
• Source type (e.g., “Court Documents”, “FOIA Responses”)

Adding Documents

Drag and drop PDF files into a collection, or click Add Files.

When you add a document, Insight:

1. Extracts the text so the AI can read it
2. Prepares it for searching
3. Stores everything locally on your computer

Sharing with Colleagues

You can share collections directly with other Insight users—no cloud upload required.

To Share

1. Open the collection
2. Click Share
3. Send the generated link to your colleague

To Join

1. Click Join Collection
2. Paste the link you received
3. Documents will copy to your computer automatically

Once synced, you both have full copies. You can work offline and still access everything.

Revoking Access

Insight uses capability-based sharing—when you share a collection, you give someone a cryptographic key that grants access. This has an important implication:

You cannot revoke access once shared.

Once someone has the key:

• They keep any documents already synced to their computer
• They can continue syncing with other peers who have the collection
• There’s no central server that can block their access

This is a fundamental tradeoff of decentralized systems: you get privacy and no single point of failure, but you lose centralized control.

If you need to stop sharing with someone:

1. Create a new collection
2. Copy the documents you want to keep sharing
3. Share the new collection only with trusted colleagues
4. Stop using the old collection

For sensitive material, consider who truly needs access before sharing. Once shared, assume the recipient has permanent access to everything in that collection.

Chat

Insight uses AI to help you find and understand information in your documents. Just ask questions in plain language—the AI does the searching for you.

How It Works

When you ask a question:

1. The AI figures out what you’re looking for
2. It searches your documents to find relevant passages
3. It reads through what it finds
4. It gives you an answer with citations

You don’t need to know which document contains the answer. Just ask, and the AI will find it.

Example Questions

Finding facts:

• “What was the total budget for 2023?”
• “Who approved the permit?”
• “When did the meeting take place?”

Summarizing:

• “Summarize the key findings from these reports”
• “What are the main arguments in this brief?”

Comparing:

• “How do the 2020 and 2023 assessments differ?”
• “Find contradictions between these statements”

Exploring:

• “What topics are covered in this collection?”
• “Find all mentions of Company X”

Citations

Every answer includes citations. Click one to jump straight to that passage in the original document—so you can verify and quote accurately.

Privacy Note

Your questions and relevant document excerpts are sent to the chat provider you configure in Settings — any of the 25+ provider families Insight supports (OpenAI, Anthropic, Gemini, Groq, Ollama for local servers, …). Your full documents stay on your computer. See Getting Started for provider setup.

Architecture

Insight is built as a Tauri 2.0 desktop application with a Rust backend and Svelte frontend.

Core Stack

Agent Architecture

User Query
    ↓
rig Agent (per request: preamble + tools + memory + max turns)
    ↓
Registry-built Model (any supported rig provider family)
    ↓
Tool Calling Loop (multi-turn)
    ↓
Synthesized Answer (with citations)

Inference is remote-only: the chat provider is configured in Settings (family + model + optional API key / base URL). Insight supports every chat-completion provider family offered by rig — OpenAI, Anthropic, Gemini, Groq, OpenRouter, and local servers like Ollama and llama.cpp — through a data-driven registry. Only the conversation—queries and retrieved excerpts—goes to the provider. Documents are stored, searched, and synced entirely on your machine.

The runtime (crates/insight-core/src/runtime.rs) builds a fresh rig agent per request: a stable preamble, the document tools, file-backed conversation memory, and a scope hook that injects the conversation’s active collections as context on every turn. generate_title and predict_next_message are bare-model completions over the same provider settings. Conversation history persists as rig messages plus a metadata sidecar under conversations/{id}/. Agent progress streams to the frontend as rig MultiTurnStreamItem JSON events.

The agent has four typed tools (search, read_chunk, list_documents, get_collection_terms) built on milli search and iroh storage. It iteratively gathers evidence to answer questions, citing sources along the way. There is no direct user-facing search—all document retrieval happens through the agent.

Data Model

Collections as Namespaces

Each collection is an iroh-docs namespace. Sharing a collection means sharing namespace access.

Namespace: 7f3a8b2c... ("Climate Research")
│
├── files/abc123/meta     → document metadata (JSON)
├── files/abc123/text     → extracted text
├── files/abc123/source   → original PDF bytes
├── files/def456/meta     → document metadata (JSON)
├── files/def456/text     → extracted text
├── files/def456/source   → original PDF bytes
├── _hash_index/{hash}    → duplicate detection index
└── _collection           → collection settings

Document Metadata

{
	"id": "abc123",
	"name": "paper.pdf",
	"file_type": "application/pdf",
	"page_count": 42,
	"tags": ["research", "climate"],
	"created_at": "2024-01-15T10:30:00Z",
	"page_boundaries": [0, 1500, 3200]
}

Content-Addressed Storage

All file content (PDFs, extracted text) is stored in iroh-blobs using content-addressing:

• Files are identified by their BLAKE3 hash
• Duplicate files are automatically deduplicated
• Content can be verified for integrity

Data Flow

Local Import

1. User adds PDF to collection
2. Store: original PDF bytes written to files/{id}/source (iroh)
3. Extract: per-page text extracted with lopdf (digital text only; page boundaries recorded), written to files/{id}/text alongside files/{id}/meta
4. Index: the watcher triggers the index worker, which chunks text (~1500 chars, ~200 overlap, page-mapped) and indexes chunks in milli for full-text search

No OCR: extraction uses lopdf only. Scanned PDFs (images of text) import fine but yield little or no extracted text and are not text-searchable.

On Sync

When document entries arrive from a peer, iroh-docs automatically syncs the entry content blobs. The SyncWatcher listens for files/*/text entries and triggers indexing:

1. Text is already available at files/{id}/text (synced by iroh)
2. Chunks are indexed in milli for search
3. Source file at files/{id}/source is available immediately

What Syncs vs What’s Local

Search is local keyword indexing over synced text: each peer builds its own milli index. Nothing beyond the raw document data is derived or replicated.

Local Storage

~/.local/share/insight/
├── iroh/               # iroh data (blobs, docs)
├── search/             # milli index
└── conversations/      # chat history

Chat provider configuration (settings v2: active provider plus per-family api_keys) lives in ~/.local/share/insight/settings.json. Conversations are stored under conversations/{id}/ as messages.json (rig message history) and meta.json (title, linked collections). On Windows, app data is under %LOCALAPPDATA%\insight\.

Development

Prerequisites

• Rust (stable)
• Node.js 20+
• pnpm
• Platform-specific dependencies (see below)

Linux

sudo apt-get install libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf

macOS

Xcode Command Line Tools:

xcode-select --install

Windows

Visual Studio Build Tools with C++ workload.

Running Locally

# Install frontend dependencies
pnpm install

# Start development mode
pnpm tauri dev

Building

# Development build
pnpm tauri build --debug

# Release build
pnpm tauri build

Testing

Backend (Rust)

cd src-tauri && cargo test

Frontend (Svelte)

pnpm test        # Watch mode
pnpm test:run    # Single run (CI)

Project Structure

insight/
├── src/                 # Svelte frontend
├── src-tauri/          # Tauri + Rust backend
│   └── src/
│       ├── commands/   # Tauri commands (IPC)
│       └── ...
├── crates/
│   └── insight-core/   # Core library
└── docs/               # This documentation

Understanding Dependencies

Prefer local tools over web searches:

# Generate and browse docs for exact dependency versions
cargo doc --open

# View dependency graph
cargo tree

# Source code at
~/.cargo/registry/src/