Encryption Guide

This guide describes how to enable and use at-rest data encryption in OpenViking.

Overview

OpenViking can encrypt newly written files with account-specific keys. The storage layer handles encryption and decryption through the existing APIs:

Transparent encryption: No API changes, application layer unaware
Multi-tenant isolation: Different accounts use independent keys
Three key providers: Local, Vault, Volcengine KMS
Backward compatible: Unencrypted old files still readable

See Data Encryption for conceptual explanations.

Encryption in Multi-Write Storage

Multi-write storage reuses the same transparent encryption model. Encryption still happens inside RAGFS, so the Python SDK, HTTP API, and CLI do not need to handle encryption or decryption directly.

Rules:

When global encryption.enabled=true, the primary backend must be encrypted.
Each backup backend may control its own encryption through encryption.enabled.
V2 multi-write metadata is stored on the raw primary and bypasses EncryptionWrappedFS.
OpenViking does not expose and does not need public encryption APIs for operating on these internal files.

See the Multi-Write Storage Guide for more multi-write configuration details.

Quick Start

1. Initialize Root Key (Local Mode)

bash

ov system crypto init-key --output-file ~/.openviking/master.key

2. Configure Encryption

Merge the following encryption settings into ~/.openviking/ov.conf; retain your model and server settings:

json

{
  "encryption": {
    "enabled": true,
    "provider": "local",
    "local": {
      "key_file": "~/.openviking/master.key"
    }
  },
  "storage": {
    "workspace": "./data"
  }
}

3. Verify

Restart the server after changing encryption settings, and run this example against that server. Install the Python SDK in the environment running the script.

python

from pathlib import Path
from openviking_sdk import SyncHTTPClient


def test():
    # OPENVIKING_API_KEY: use a tenant-bound user/admin key when authentication is enabled.
    client = SyncHTTPClient(url="http://localhost:1933")
    try:
        client.initialize()
        sample = Path("./encrypted-sample.txt")
        sample.write_text("Hello, encrypted world!", encoding="utf-8")
        imported = client.add_resource(
            path=str(sample),
            wait=True,
            timeout=120,
        )
        results = client.find(
            query="encrypted", target_uri=imported["root_uri"]
        )
        print(f"Found {len(results.get('resources', []))} resources")
    finally:
        client.close()


test()

It waits for import processing and checks retrieval; successful retrieval alone does not prove encryption at rest. Use Check File Content to inspect the stored file header.

API Key Hashing Configuration

File encryption and API key hashing are independent controls:

Encryption Layer	Config	Algorithm	Reversible	Description
File Layer	`encryption.enabled`	AES-GCM	Yes	Protects entire storage files
API Key Field Layer	`encryption.api_key_hashing.enabled`	Argon2id	No	Protects API keys themselves

Default Behavior

By default, encryption.api_key_hashing.enabled = false:

API keys are stored in plaintext within JSON files
If encryption.enabled = true, the entire file is protected by AES-GCM encryption
ov admin list-users can display the full API key
If encryption.enabled = true and API key hashing stays disabled, the server writes an informational log line about this at startup

Enabling Argon2id Hashing

Enable Argon2id one-way hashing to stop storing recoverable API key values:

json

{
  "encryption": {
    "enabled": true,
    "api_key_hashing": {
      "enabled": true
    }
  }
}

Note: When enabled:

API keys are stored using Argon2id one-way hashing
Plaintext keys cannot be recovered from hash values
ov admin list-users only shows key_prefix instead of the full API key
Plaintext keys are only visible when creating users or regenerating keys

Configuration Example

json

{
  "encryption": {
    "enabled": true,
    "provider": "local",
    "local": {
      "key_file": "~/.openviking/master.key"
    },
    "api_key_hashing": {
      "enabled": false
    }
  }
}

Choosing a Key Provider

Provider	Use Case	Pros	Cons
Local	Dev environments, single-node	Simple, no external services	Requires local file permissions and separate key backups
Vault	Production, multi-cloud	Enterprise-grade KMS, version control	Requires deploying and maintaining Vault
Volcengine KMS	Volcengine cloud	Cloud-native KMS service	Volcengine-only

Local Mode Detailed Guide

Initialize Root Key

bash

# Generate and save to specified path
ov system crypto init-key --output-file ~/.openviking/master.key

# Or use short option
ov system crypto init-key -f ~/.openviking/master.key

Output example:

✓ Root key generated successfully
✓ Saved to: /Users/you/.openviking/master.key

Security Tips

⚠️ Keep master.key safe
Recommend setting file permissions to 600 (owner-only read/write)
Regularly back up the key file
Don't commit the key file to version control

Configuration Example

json

{
  "encryption": {
    "enabled": true,
    "provider": "local",
    "local": {
      "key_file": "~/.openviking/master.key"
    }
  }
}

Vault Mode Detailed Guide

Prerequisites

HashiCorp Vault service deployed; install the Python hvac dependency in the server environment
Transit engine enabled
Vault Token with sufficient permissions

Configure Vault

Enable Transit engine (if not already enabled):

bash

vault secrets enable transit

Enable KV engine (if not already enabled):

bash

# KV v2 (recommended)
vault secrets enable -path=secret -version=2 kv

# Or KV v1
vault secrets enable -path=secret -version=1 kv

Use one KV command matching your deployment, not both. The example below uses the secret mount with KV v2. Pre-create the Transit key with an administrative token:

bash

vault write -f transit/keys/openviking-root-key type=aes256-gcm96

Configure OpenViking:

json

{
  "encryption": {
    "enabled": true,
    "provider": "vault",
    "vault": {
      "address": "https://vault.example.com:8200",
      "token": "hvs.xxxxxxxxxxxxxxxxxxxxx",
      "mount_point": "transit",
      "kv_mount_point": "secret",
      "kv_version": 2,
      "root_key_name": "openviking-root-key",
      "encrypted_root_key_key": "openviking-encrypted-root-key"
    }
  }
}

Configuration Parameters:

Parameter	Description	Default
`address`	Vault server address	Required
`token`	Vault authentication token	Required
`mount_point`	Transit engine mount path	`"transit"`
`kv_mount_point`	KV engine mount path	`"secret"`
`kv_version`	KV engine version (1 or 2)	`1`
`root_key_name`	Key name in Transit engine	`"openviking-root-key"`
`encrypted_root_key_key`	Path to store encrypted root key in KV engine	`"openviking-encrypted-root-key"`

Vault Permission Recommendations

For the configuration above, the service token needs Transit key metadata access, encrypt/decrypt access, and KV access to store the wrapped root key. With the engines and Transit key pre-created, a policy example is:

hcl

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

path "transit/keys/openviking-root-key" {
  capabilities = ["read"]
}

path "secret/data/openviking-encrypted-root-key" {
  capabilities = ["read", "create", "update"]
}

path "transit/encrypt/openviking-root-key" {
  capabilities = ["update"]
}

path "transit/decrypt/openviking-root-key" {
  capabilities = ["update"]
}

For KV v1, replace secret/data/openviking-encrypted-root-key with secret/openviking-encrypted-root-key. The provider also probes sys/mounts; denial is logged as a warning and does not stop startup when Transit is already enabled.

Volcengine KMS Mode Detailed Guide

Prerequisites

Volcengine KMS service activated
Symmetric key created
Valid Access Key and Secret Key

Create KMS Key

Visit Volcengine KMS Console
Click "Create Key"
Select "Symmetric Key", algorithm AES_256
Record the Key ID

Configure OpenViking

json

{
  "encryption": {
    "enabled": true,
    "provider": "volcengine_kms",
    "volcengine_kms": {
      "key_id": "d926aa0d-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "region": "cn-beijing",
      "access_key": "AKLTxxxxxxxxxxxxxxxxxx",
      "secret_key": "Tmpxxxxxxxxxxxxxxxxxxxxxx",
      "endpoint": null,
      "key_file": "~/.openviking/openviking-volcengine-root-key.enc"
    }
  }
}

Configuration Parameters:

Parameter	Description	Default
`key_id`	KMS Key ID	Required
`region`	Region	Required
`access_key`	Access Key	Required
`secret_key`	Secret Key	Required
`endpoint`	Custom KMS endpoint (optional)	`null` (use default endpoint)
`key_file`	Local cache file path for encrypted root key	`"~/.openviking/openviking-volcengine-root-key.enc"`

Permission Recommendations

Configure minimal permissions for the Access Key:

json

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt"
      ],
      "Resource": [
        "trn:kms:*:*:key/d926aa0d-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      ]
    }
  ]
}

Verifying Encryption

Method 1: Check File Content

Inspect the physical backend file, not the API response, which is decrypted. Substitute its actual path below. Encrypted files start with magic number OVE1:

bash

# View first 4 bytes
hexdump -C ./data/agfs/your-file | head -1

Encrypted file:

00000000  4f 56 45 31 01 01 00 00  00 20 8a 7b 2c 9d 1e  |OVE1..... .{,..|

(First 4 bytes are 4f 56 45 31 = "OVE1")

Unencrypted file:

00000000  7b 22 63 6f 6e 74 65 6e  74 73 22 3a 5b 7b 22 70  |{"contents":[{"p|

Method 2: Restart and Read Back

Read the same imported file through the API before and after restarting the server. Verify its content matches and inspect the physical backend file header as above. This checks that the deployment reloads the expected key and can read its ciphertext.

Do not treat an arbitrary exception in an internal provider call as an encryption test: argument errors, missing dependencies, and network errors can all fail before decryption.

Migration Guide

Migrating from Unencrypted to Encrypted

Enabling encryption does not rewrite existing plaintext files. They remain readable for backward compatibility, while new writes use encryption. To encrypt existing public scopes, migrate them through OVPack into a new, empty encrypted storage environment:

Stop application writes and create a logical backup while the original unencrypted environment is running:

bash

ov backup ./backups/before-encryption.ovpack

Stop OpenViking. Enable encryption and point the storage configuration at a new, empty workspace/backend. Keep the original data and encryption key backup until verification is complete.
Start the encrypted environment. In API key mode, first create the target account and a restore operator with an admin key, then point the CLI at the target using that key, as described in Full Backup and Restore. Restore writes the package content through the encrypted storage layer:

Account initialization creates scope directories, so fail would reject this restore. Confirm that the target contains only the newly created account's initial content before using overwrite below. If it already contains business data, stop and follow the target-backup and conflict-review steps in the OVPack guide instead.

bash

ov restore ./backups/before-encryption.ovpack --on-conflict overwrite

Verify resource, user, session, and index data before switching traffic. OVPack excludes runtime/internal state such as queues, uploads, locks, and watches; recreate or validate those separately.

See OVPack Import and Export for supported scopes and restore options.

Switching Key Providers

Keep the original provider and key available while exporting a logical OVPack backup. Restore it into a separate empty deployment configured with the new provider, following the migration steps above. Verify contents, access permissions, and rebuilt indexes before switching traffic. Changing the provider setting in place does not re-encrypt existing files.

Troubleshooting

Key File Not Found

Error: Key file not found: ~/.openviking/master.key

Solution:

Check file path is correct
Use absolute path
Ensure ~ is properly expanded (use expanduser())

Vault Connection Failed

Error: Failed to connect to Vault

Solution:

Check if Vault service is running
Verify address configuration
Check network connectivity and firewall
Confirm Token is valid and not expired

Volcengine KMS Authentication Failed

Error: Invalid credentials

Solution:

Check Access Key and Secret Key are correct
Confirm key has sufficient permissions
Verify region configuration is correct

Existing ciphertext cannot be read

Check that the deployment still has the original root key, provider settings, account identity, and intact ciphertext. For Vault, retain both the Transit key and the KV entry containing the wrapped root key; for KMS, retain the KMS key and local wrapped-key file. Do not generate a replacement root key to repair access to existing ciphertext.

Partial reads load and authenticate the full encrypted file, then return the requested plaintext slice. If a partial read returns ciphertext, record the server version and compare full and partial reads on a copied test file. Check the storage encryption configuration and relevant fixes before upgrading, and keep the original data and keys available for rollback.

Data Encryption - Encryption concepts
Configuration Guide - Complete configuration reference
Multi-Tenant - Account, user, and agent isolation model

Vault references: KV v2, Transit API.

Encryption Guide ​

Overview ​

Encryption in Multi-Write Storage ​

Quick Start ​

1. Initialize Root Key (Local Mode) ​

2. Configure Encryption ​

3. Verify ​

API Key Hashing Configuration ​

Default Behavior ​

Enabling Argon2id Hashing ​

Configuration Example ​

Choosing a Key Provider ​

Local Mode Detailed Guide ​

Initialize Root Key ​

Security Tips ​

Configuration Example ​

Vault Mode Detailed Guide ​

Prerequisites ​

Configure Vault ​

Vault Permission Recommendations ​

Volcengine KMS Mode Detailed Guide ​

Prerequisites ​

Create KMS Key ​

Configure OpenViking ​

Permission Recommendations ​

Verifying Encryption ​

Method 1: Check File Content ​

Method 2: Restart and Read Back ​

Migration Guide ​

Migrating from Unencrypted to Encrypted ​

Switching Key Providers ​

Troubleshooting ​

Key File Not Found ​

Vault Connection Failed ​

Volcengine KMS Authentication Failed ​

Existing ciphertext cannot be read ​

Related Documentation ​

Encryption Guide

Overview

Encryption in Multi-Write Storage

Quick Start

1. Initialize Root Key (Local Mode)

2. Configure Encryption

3. Verify

API Key Hashing Configuration

Default Behavior

Enabling Argon2id Hashing

Configuration Example

Choosing a Key Provider

Local Mode Detailed Guide

Initialize Root Key

Security Tips

Configuration Example

Vault Mode Detailed Guide

Prerequisites

Configure Vault

Vault Permission Recommendations

Volcengine KMS Mode Detailed Guide

Prerequisites

Create KMS Key

Configure OpenViking

Permission Recommendations

Verifying Encryption

Method 1: Check File Content

Method 2: Restart and Read Back

Migration Guide

Migrating from Unencrypted to Encrypted

Switching Key Providers

Troubleshooting

Key File Not Found

Vault Connection Failed

Volcengine KMS Authentication Failed

Existing ciphertext cannot be read

Related Documentation