Architecture Overview

OpenViking is a context database designed for AI Agents, unifying all context types (Memory, Resource, Skill) into a directory structure with semantic retrieval and progressive content loading.

System Overview

Text version

CLI / SDK / HTTP client
          |
      HTTP Server
          |
      Service Layer
          |
   +------+------+----------------+
   |             |                |
Retrieval     Sessions       Resource / Skill import
   |             |                |
   |       Memory extraction  Parse / Semantic queues
   |             |                |
   +-------------+----------------+
                 |
             VikingFS
           /          \
      RAGFS         Vector index

Core Modules

Module	Responsibility	Key Capabilities
Client	Unified entry	Sends supported SDK/CLI operations to the HTTP API
Service	Business logic	FSService, SearchService, SessionService, ResourceService, PackService, DebugService
Retrieve	Context retrieval	Intent analysis (IntentAnalyzer), global retrieval (HierarchicalRetriever), Rerank
Session	Session management	Message recording, usage tracking, session archiving, triggering memory commit
Parse	Context extraction	Document parsing (PDF/MD/HTML), tree building (TreeBuilder), async semantic generation
Memory	Memory extraction	Schema-driven extraction by MemoryType (ExtractLoop), LLM merge and deduplication written back as patches (MemoryUpdater); orchestrated by SessionCompressorV3
Storage	Storage layer	VikingFS virtual filesystem, vector index, RAGFS integration

Service Layer

The Service layer decouples business logic from the transport layer. The CLI and SDKs reach it through the HTTP Server:

Service	Responsibility	Key Methods
FSService	File system operations	ls, mkdir, rm, mv, tree, stat, read, abstract, overview, grep, glob
SearchService	Semantic search	search, find
SessionService	Session management	session, sessions, commit, delete
ResourceService	Resource import	add_resource, add_skill, wait_processed
PackService	Import/export and backup/restore	export_ovpack, import_ovpack, backup_ovpack, restore_ovpack
DebugService	Debug service	observer (ObserverService)

Dual-Layer Storage

OpenViking uses a dual-layer storage architecture separating content from index (see Storage Architecture):

Layer	Responsibility	Content
RAGFS	Content storage	L0/L1/L2 full content, multimedia files
Vector Index	Index storage	URIs, vectors, metadata, and text used for retrieval, including abstracts

Data Flow Overview

Adding Context

Input → Parser → TreeBuilder → ResourceProcessor → RAGFS → SemanticQueue → Vector Index

1. Parser: Parse source documents into files and directories; model use depends on the selected parser
2. TreeBuilder: Resolve the target URI and retain temporary artifact references
3. ResourceProcessor: Persist content in RAGFS and enqueue semantic processing
4. SemanticQueue: Async bottom-up L0/L1 generation
5. Vector Index: Build index for semantic search

Retrieving Context

Query → Intent Analysis → Global Retrieval → Rerank → Results

1. Intent Analysis: Analyze query intent, generate 0-5 typed queries
2. Global Retrieval: One vector search per query within the permitted scope
3. Rerank: Optional single pass over the recalled candidates
4. Results: Return contexts sorted by relevance

Session Commit

Messages → Archive Boundary → Archive → Memory Extraction → Storage

1. Messages: Accumulate conversation messages and usage records
2. Archive Boundary: Split archived and retained messages according to the commit parameters; by default, archive all current messages
3. Archive: Generate L0/L1 for history segments
4. Memory Extraction: Extract memories from messages according to the memory policy and MemoryType schemas
5. Storage: Write to RAGFS + vector index

Deployment Mode

HTTP Mode

For team sharing, production deployment, and cross-language integration:

# Python SDK connects to OpenViking Server
from openviking_sdk import SyncHTTPClient

client = SyncHTTPClient(url="http://localhost:1933", api_key="your-key")

# Or use curl / any HTTP client
curl http://localhost:1933/api/v1/search/find \
  -H "X-API-Key: your-key" \
  -H "Content-Type: application/json" \
  -d '{"query": "how to use openviking"}'

• Server runs as standalone process (openviking-server)
• Clients connect via HTTP API
• Supports any language that can make HTTP requests
• See Server Deployment for setup

Design Principles

Principle	Description
Pure Storage Layer	Storage only handles RAGFS operations and basic vector search; Rerank is in retrieval layer
Three-Layer Information	L0/L1/L2 enables progressive detail loading, saving token consumption
Two-Stage Retrieval	Vector search recalls candidates + Rerank improves accuracy
Single Data Source	RAGFS holds source files; vector records retain the text and metadata needed for retrieval

Related Documents

• Context Types - Resource/Memory/Skill types
• Context Layers - L0/L1/L2 model
• Viking URI - Unified resource identifier
• Storage Architecture - Dual-layer storage details
• Retrieval Mechanism - Retrieval process details
• Context Extraction - Parsing and extraction process
• Session Management - Session and memory management
• Transaction Model - Write and consistency model
• Data Encryption - At-rest encryption and key architecture
• Multi-Tenant - Account, user, and peer isolation model
• Metrics - /metrics usage and key metric explanations
• Privacy Configs and Skill Privacy Extraction/Restore - Versioning, placeholder extraction, and read-time restore