Skip to content

Developer Documentation

Welcome to the cbintel developer documentation. This guide explains the system architecture from the ground up, covering network infrastructure, primitives, jobs, workspaces, graphs, and the chat-to-execution pipeline.

Layer Description Key Concepts
CLI User Guide Interactive CLI Commands, templates, auto mode
Architecture System overview Component relationships, data flow
Network Network infrastructure OpenWRT, VPN, Tor, GeoRouter
Primitives Core operations AI clients, HTTP, browser, vectors
Jobs Job system Workers, lifecycle, output shapes
Workspaces Organization Artifacts, runs, manifests
Graphs DAG execution YAML schema, operations, templates
Type System Type safety Hierarchy, compatibility, filters
Chat Pipeline Chat to execution Intent, graph building, auto-complete
Roadmap Future vision Near-term, mid-term, long-term

System Architecture

graph TB
    subgraph "User Interface"
        CLI[CLI<br/>npm run dev]
        CHAT[Research Agent<br/>Natural Language]
        API[REST API<br/>intel.nominate.ai]
    end

    subgraph "Orchestration"
        INTENT[Intent Classifier]
        GRAPH[Graph Executor]
        QUEUE[Redis Job Queue]
    end

    subgraph "Workers"
        W_CRAWL[CrawlWorker]
        W_GRAPH[GraphWorker]
        W_BROWSER[BrowserWorker]
        W_OTHER[... 4 more]
    end

    subgraph "Primitives"
        AI[AI Clients]
        HTTP[HTTP/Fetch]
        FERRET[Ferret/SWRM]
        VECTL[Vector Store]
    end

    subgraph "Network Layer"
        VPN[VPN Banks<br/>16 OpenWRT Workers]
        TOR[Tor Gateway<br/>tor.nominate.ai]
        GEO[GeoRouter]
    end

    CLI --> API
    CHAT --> INTENT --> GRAPH
    API --> QUEUE --> W_CRAWL & W_GRAPH & W_BROWSER & W_OTHER
    W_CRAWL & W_GRAPH --> AI & HTTP & VECTL
    HTTP --> GEO --> VPN
    HTTP --> TOR
    W_BROWSER --> FERRET --> GEO

Getting Started

Quick Start with CLI

The fastest way to explore cbintel is through the interactive CLI:

cd cli
npm install
npm run dev

Then try these commands: - Type a query: Research AI safety trends - Use a template: /run person_deep_research "Elon Musk" - Run experiments: /auto

See the CLI User Guide for complete documentation.

For New Developers

  1. Start with Architecture to understand the overall system
  2. Read Network to understand the proxy infrastructure
  3. Explore Primitives for core operations
  4. Study Graphs for the research execution model

For API Consumers

  1. Review Jobs for job submission and tracking
  2. Check API Reference for endpoint details
  3. See Workspaces for organizing research

For Researchers

  1. Understand Graphs for building research pipelines
  2. Learn the Type System for data flow
  3. Explore Chat Pipeline for natural language queries

Package Structure

src/cbintel/
├── ai/           # AI client wrappers (Anthropic, Ollama, CBAI)
├── net/          # Network operations (HTTP, search, URL cleaning)
├── io/           # File/process I/O (HTML, markdown, storage)
├── cluster/      # VPN cluster management API
├── geo/          # Geographic VPN routing
├── tor/          # Tor gateway client
├── ferret/       # Browser automation (SWRM)
├── crawl/        # AI-powered crawl pipeline
├── lazarus/      # Historical web archives
├── vectl/        # Vector embeddings and search
├── screenshots/  # Browser automation (Playwright)
├── jobs/         # Async job queue with workers
├── workspace/    # Workspace management
└── graph/        # Graph execution engine

Key Technologies

Technology Purpose Module
Claude API AI evaluation, synthesis cbintel.ai
Ollama Local LLM, embeddings cbintel.ai, cbintel.vectl
Playwright Browser automation cbintel.screenshots
OpenWRT/LuCI VPN worker control cbintel.cluster
Redis Job queue cbintel.jobs
FastAPI REST APIs cbintel.jobs, cbintel.cluster

Environment Setup

# Activate Python environment
source ~/.pyenv/versions/nominates/bin/activate

# Install with dev dependencies
pip install -e ".[dev]"

# Install Playwright browsers
playwright install

# Run tests
pytest

API Endpoints

Domain Port Service
intel.nominate.ai 9003 Jobs API
network.nominate.ai 32203 VPN Cluster API
tor.nominate.ai - Tor Gateway API
index.nominate.ai - Content Indexing API
ai.nominate.ai - AI/LLM API
geo.nominate.ai - Geocoding API