Skip to content

Data Encryption ​

OpenViking supports at-rest encryption: it encrypts files before storage and decrypts them for authorized reads. Each account uses a separate account key.

Overview ​

Why Encryption ​

Multiple accounts can share an AGFS instance. When encryption is enabled:

  • Encrypted files require the corresponding keys to decrypt; protect keys separately from data
  • Different accounts' data is encrypted with independent keys for tenant isolation
  • The RAGFS encryption wrapper handles file reads and writes at runtime; coverage depends on each backend configuration

Transparency ​

Enabling encryption preserves the client API:

  • No client API changes: Existing code works without modification
  • Plaintext responses: Authorized reads still return decrypted content; unavailable keys or failed ciphertext authentication cause errors
  • Compatible with existing files: Old plaintext files remain readable; enabling encryption does not encrypt them automatically

Three-Layer Key Architecture ​

OpenViking uses an Envelope Encryption architecture with a three-layer key system:

Three-layer key hierarchyThe root key, unique per instance and stored locally or wrapped by KMS or Vault, derives one account key per account with HKDF-SHA256. The account key wraps a random per-write file key with AES-256-GCM; the wrapped file key is stored in the file envelope and encrypts the content.HKDF-SHA256 per accountRoot Keyone per OpenViking instancelocal file, or KMS / Vault-wrappedAES-256-GCM key wrapAccount Key (KEK)one per account, derived at runtime,never storedFile Key (DEK)random per write, stored wrappedin the file envelope; encrypts content

Key Hierarchy Summary ​

LayerNameDescriptionQuantity
Root KeyRoot KeySystem master key, used to derive all account keys1 per instance
Account KeyAccount KeyIndependent key per account, derived from root key1 per account
File KeyFile KeyOne-time random key per file1 per write

Key Providers ​

OpenViking supports three key providers for different deployment scenarios:

ProviderUse CaseRoot Key StorageFeatures
LocalDev environments, single-node deploymentsLocal file ~/.openviking/master.keySimple, no external services
VaultProduction, multi-cloudTransit-encrypted value stored in Vault KVVault protects the root key
Volcengine KMSVolcengine cloud deploymentsKMS-encrypted value stored in local key_fileKMS protects the root key

Local (File) ​

Suitable for development and single-node deployments:

json
{
  "encryption": {
    "enabled": true,
    "provider": "local",
    "local": {
      "key_file": "~/.openviking/master.key"
    }
  }
}

Initialization command:

bash
ov system crypto init-key --output-file ~/.openviking/master.key

Vault (HashiCorp Vault) ​

Suitable for production and multi-cloud deployments:

json
{
  "encryption": {
    "enabled": true,
    "provider": "vault",
    "vault": {
      "address": "https://vault.example.com:8200",
      "token": "hvs.your-vault-token",
      "mount_point": "transit",
      "kv_mount_point": "secret",
      "kv_version": 1,
      "root_key_name": "openviking-root-key",
      "encrypted_root_key_key": "openviking-encrypted-root-key"
    }
  }
}

Volcengine KMS ​

Suitable for Volcengine cloud deployments:

json
{
  "encryption": {
    "enabled": true,
    "provider": "volcengine_kms",
    "volcengine_kms": {
      "key_id": "d926aa0d-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "region": "cn-beijing",
      "access_key": "AKLTxxxxxxxxxxxxxxxxxx",
      "secret_key": "Tmpxxxxxxxxxxxxxxxxxxxxxx",
      "endpoint": null,
      "key_file": "~/.openviking/openviking-volcengine-root-key.enc"
    }
  }
}

How It Works ​

Startup resolves the root key, then RAGFS EncryptionWrappedFS handles file content for protected backends. Python key providers load or unwrap the root key; individual file operations do not call KMS to derive account keys.

Write Flow ​

Encrypted write pathThe client writes plaintext. The RAGFS encryption wrapper derives the account key, generates a random file key and nonces, encrypts the content with AES-256-GCM, wraps the file key with the account key, and writes the OVE1 envelope to the backend.ClientEncryption wrapperRAGFSAccount keyfrom root keyBackendlocal / S3write(uri, plaintext)derive (HKDF, cached)account key1. random File Key + nonces2. AES-256-GCM encrypt content3. wrap File Key with account key4. build OVE1 envelopewrite envelope (ciphertext)storedok
  1. Derive the account key from the root key and account_id with HKDF-SHA256; it may be cached at runtime.
  2. Generate a random File Key and nonces for this encryption.
  3. Encrypt content with the File Key using AES-256-GCM, then wrap the File Key with the account key.
  4. Persist the envelope header, wrapped File Key, nonces, and content ciphertext.

Read Flow ​

Encrypted read pathThe client reads a URI. The wrapper fetches the stored bytes; files without the OVE1 magic are returned as plaintext. Otherwise it derives the account key, unwraps the file key, authenticates and decrypts the content, and returns plaintext or an error.ClientEncryption wrapperRAGFSAccount keyfrom root keyBackendlocal / S3read(uri)read bytesstored bytesno OVE1 magic → return as plaintextderive (HKDF, cached)account key1. unwrap File Key with account key2. authenticate + decrypt (AES-GCM)wrong key or bad tag → errorplaintext

For encrypted files, the account key unwraps the File Key, which authenticates and decrypts the content. Old files without OVE1 are read as plaintext. An incorrect key or failed ciphertext authentication returns an error, not decrypted content.

Envelope Format ​

Encrypted files use a unified envelope format starting with the magic number OVE1 (OpenViking Encryption v1):

FieldSizeValue
Magic4 bytesOVE1
Version1 byte0x01
Provider1 byte0x01 = local
Encrypted File KeyvariableFile Key wrapped with the account key
…nonces and content ciphertext
  • If a file doesn't start with OVE1, it's treated as unencrypted and plaintext is returned directly
  • Old files remain readable; protecting existing plaintext requires a separate migration or rewrite

Multi-Tenant Isolation ​

Different accounts' data is encrypted with independent Account Keys:

  • Account A's key cannot decrypt Account B's files
  • For files already encrypted, obtaining backend ciphertext still requires the corresponding key to decrypt it
  • Separate keys supplement tenant access controls; they do not replace authentication or storage permissions

Encryption Coverage ​

This protects file backends where encryption is enabled. It does not automatically encrypt a separate vector database, logs, or exported OVPack files; vector records can contain abstracts and memory text. Multi-write backups can disable encryption individually, so check each backend configuration.

Configuration Example ​

See Configuration Guide for detailed configuration.

Open source under the AGPL-3.0 License. Font licenses