v6.0.0 Release

HIGH
PERFORMANCE

The acid-compliant, multi-process safe JSON database for Node.js. Powered by a Rust core via N-API.

2.5M+
Reads/Sec
545k+
Writes/Sec
0.002ms
Latency

Introduction

Welcome to the future of Node.js data storage. jsondb-high bridges the gap between the simplicity of JSON files and the performance of enterprise-grade databases.

New in v6.0: MVCC concurrency, mmap O(1) startup, .explain() query introspection, PITR, pluggable storage adapters, WAL replication, zero-copy streams, WASM fallback, and lite mode.

Features

โšก Blazing Fast

Core logic in Rust via N-API. Up to 2M ops/s reads.

๐Ÿ”’ Multi-Process Safe

OS-level advisory file locking prevents corruption.

๐Ÿงต Multi-Core

Adaptive parallelism utilizing all CPU cores with Rayon.

๐Ÿ›ก๏ธ ACID Durability

Write-Ahead Logging (WAL) ensures your data is safe.

Installation

# Using bun (recommended)
bun add jsondb-high

# Using npm
npm install jsondb-high
Note: v6 ships prebuilt binaries for most platforms. If no prebuild is available, it builds from source automatically. If Rust is not installed, it falls back to WASM.

Requirements

Quick Start

import JSONDatabase from 'jsondb-high';

const db = new JSONDatabase('db.json');

// Write
await db.set('user.1', { name: 'Alice', role: 'admin' });

// Read
const user = await db.get('user.1');
console.log(user); // { name: 'Alice', role: 'admin' }

Hybrid Architecture

jsondb-high offers multiple storage and safety modes. Choose based on your performance and durability needs.

Durability Modes

const db = new JSONDatabase('db.json', {
  durability: 'batched',   // 'none' | 'lazy' | 'batched' | 'sync'
  walFlushMs: 10,          // Sync every 10ms (Group Commit)
  lockMode: 'exclusive'
});
ModeThroughputLatencyDurability Window
none~545k ops/s0.002msManual save only
lazy~400k ops/s0.003ms100ms
batched~525k ops/s0.002ms10ms (Recommended)
sync~2k ops/s0.5msImmediate

Locking Modes

Prevent corruption from multiple processes using the same file:

const db = new JSONDatabase('db.json', {
  lockMode: 'exclusive' // 'exclusive' | 'shared' | 'none'
});

O(1) Indexing

const db = new JSONDatabase('db.json', {
  indices: [{ name: 'email', path: 'users', field: 'email' }]
});

// Instant Lookup
const user = await db.findByIndex('email', 'alice@corp.com');

Parallel Processing

The database automatically detects available CPU cores and uses parallel processing for large datasets (โ‰ฅ100 items). Falls back to efficient single-threaded operation for small workloads to avoid overhead.

const info = db.getSystemInfo();
console.log(info);
// {
//   availableCores: 8,
//   parallelEnabled: true,
//   recommendedBatchSize: 1000
// }

DBOptions Interface

interface DBOptions {
  indices?: IndexConfig[];           // Array of index configurations
  wal?: boolean;                     // Enable Write-Ahead Logging
  encryptionKey?: string;            // AES-256-GCM encryption key (32 chars)
  autoSaveInterval?: number;         // Auto-save interval in ms
  lockMode?: 'exclusive' | 'shared' | 'none';  // Process locking mode
  lockTimeoutMs?: number;            // Lock timeout in ms
  durability?: 'none' | 'lazy' | 'batched' | 'sync';  // Durability mode
  walBatchSize?: number;             // WAL batch size
  walFlushMs?: number;               // WAL flush interval in ms
  schemas?: Record<string, Schema>;  // Path-based schemas
  slowQueryThresholdMs?: number;     // Slow query threshold in ms
  stripeCount?: number;              // v6: DashMap concurrency stripes
  bufferPoolSizeMB?: number;         // v6: Buffer pool size for mmap
  bufferPageSizeKB?: number;         // v6: Page size for buffer pool
}

Basic Operations

await db.set('config.theme', 'dark');
const val = await db.get('config.theme', 'light');
if (await db.has('users.1')) { ... }
await db.delete('users.1.settings');

Array Operations

await db.push('users.1.tags', 'premium', 'beta');
await db.pull('users.1.tags', 'beta');

Math Operations

const newCount = await db.add('users.1.loginCount', 1);
const newCredits = await db.subtract('users.1.credits', 50);

Query Builder

const results = await db.query('users')
  .where('age').gt(18)
  .where('role').eq('admin')
  .limit(10)
  .sort({ age: -1 })
  .select(['id', 'name'])
  .exec();

Aggregations

const count = db.query('users').count();
const total = db.query('orders').sum('amount');
const grouped = db.query('users').groupBy('department');

Find Operations

const user = await db.find('users', u => u.age > 18);
const adults = await db.findAll('users', u => u.age >= 18);

Pagination

const page = await db.paginate('users', 1, 20);

Batch & Parallel

await db.batch([
  { type: 'set', path: 'logs.1', value: 'log data' },
  { type: 'delete', path: 'temp.cache' }
]);

// Parallel (High Performance)
await db.batchSetParallel(largeArrayOfOperations);

Transactions

await db.transaction(async (data) => {
  if (data.bank.balance >= 100) {
    data.bank.balance -= 100;
    data.users['1'].wallet += 100;
  }
  return data;
});

Snapshots

const backupPath = await db.createSnapshot('daily');
await db.restoreSnapshot(backupPath);

TTL (Time to Live)

await db.setWithTTL('session.abc', { id: 1 }, 60);

Pub/Sub

const unsubscribe = db.subscribe('users.*', (newVal) => {
  console.log('Changed:', newVal);
});

Encryption

const db = new JSONDatabase('secure.json', {
  encryptionKey: 'your-32-character-secret-key!!'
});

Middleware

db.before('set', 'users.*', (ctx) => {
  ctx.value.updatedAt = Date.now();
  return ctx;
});

Utility Methods

await db.keys('users');
await db.values('users');
await db.count('users');
await db.clear();
await db.stats();
await db.save(); // Force save
await db.sync(); // Durability sync

Events

db.on('change', ({ path, value, oldValue }) => { ... });
db.on('batch', ({ operations }) => { ... });
db.on('transaction:commit', () => { ... });
db.on('transaction:rollback', ({ error }) => { ... });
db.on('snapshot:created', ({ path, name }) => { ... });
db.on('snapshot:restored', ({ path }) => { ... });
db.on('ttl:expired', ({ path }) => { ... });
db.on('error', (error) => { ... });

Schema Validation (v5.1+)

Define schemas to enforce data structure and validation rules at specific paths.

const db = new JSONDatabase('db.json', {
  schemas: {
    'users': {
      type: 'object',
      properties: {
        id: { type: 'number' },
        email: { 
          type: 'string', 
          pattern: '^[\\w.-]+@[\\w.-]+\\.\\w+$' 
        },
        age: { 
          type: 'number', 
          minimum: 0, maximum: 150 
        },
        role: { 
          type: 'string', 
          enum: ['admin', 'user', 'guest'] 
        },
        tags: {
          type: 'array',
          items: { type: 'string' },
          uniqueItems: true
        }
      },
      required: ['id', 'email']
    }
  }
});

// โŒ Missing required field
await db.set('users.1', { id: 1 });

// โœ… Valid data
await db.set('users.1', { 
  id: 1, 
  email: 'alice@example.com',
  age: 25,
  role: 'admin'
});

Schema Types & Constraints

TypeConstraints
stringminLength, maxLength, pattern (regex)
numberminimum, maximum, exclusiveMinimum, exclusiveMaximum
arrayminItems, maxItems, uniqueItems, items (item schema)
objectproperties, required
All typesenum (allowed values)

Where Clauses

.where('field').eq(value)      // Equal
.where('field').ne(value)      // Not equal
.where('field').gt(value)      // Greater than
.where('field').gte(value)     // Greater or equal
.where('field').lt(value)      // Less than
.where('field').lte(value)     // Less or equal
.where('field').between(1, 10) // Between range
.where('field').in([1, 2])     // In array
.where('field').notIn([1])    // Not in array
.where('field').contains('x')  // String contains
.where('field').startsWith('x')// String starts with
.where('field').endsWith('x')  // String ends with
.where('field').matches(/^x/) // Regex match
.where('field').exists()       // Field exists
.where('field').isNull()       // Is null
.where('field').isNotNull()    // Is not null

Parallel Query

High-performance filtering using native Rust parallel iteration.

const activeAdults = await db.parallelQuery('users', [
  { field: 'age', op: 'gte', value: 18 },
  { field: 'status', op: 'eq', value: 'active' }
]);

Parallel Aggregation

const count = await db.parallelAggregate('orders', 'count');
const totalRevenue = await db.parallelAggregate('orders', 'sum', 'amount');
const avgOrder = await db.parallelAggregate('orders', 'avg', 'amount');

Parallel Joins (Lookups)

const usersWithOrders = await db.parallelLookup(
  'users',      // left collection
  'orders',     // right collection
  'id',         // left field (users.id)
  'userId',     // right field (orders.userId)
  'orders'      // output field name
);

๐Ÿงฌ MVCC Concurrency (v6)

Multi-Version Concurrency Control ensures readers never block writers. Concurrent writes to the same key return a ConflictError instead of deadlocking.

const db = new JSONDatabase('db.json', {
  stripeCount: 256,  // DashMap auto-scales with CPU cores
});

// These proceed in parallel โ€” different collections, no locking
await Promise.all([
  db.set('users.1', { name: 'Alice' }),
  db.set('orders.1', { total: 99 }),
  db.set('logs.1', { event: 'login' }),
]);

๐Ÿ“‚ mmap O(1) Startup (v6)

Database files are memory-mapped via memmap2 โ€” near-instant startup regardless of file size.

const db = new JSONDatabase('large.json', {
  bufferPoolSizeMB: 128,   // Configurable buffer pool
  bufferPageSizeKB: 32,    // Page size tuning
});
// 100MB file? Opens in <1ms via mmap

๐Ÿ”ฌ .explain() API (v6)

Inspect query execution plans โ€” no more guessing what the Rust engine is doing:

const plan = await db.query('users')
  .where('age').gt(18)
  .sort({ age: -1 })
  .limit(10)
  .explain();

// {
//   scanType: 'FILTER_SCAN',
//   collectionSize: 50000,
//   filtersApplied: [{ field: 'age', op: 'Gt' }],
//   sortApplied: [{ field: 'age', direction: 'desc' }],
//   limit: 10,
//   matchedCount: 1234,
//   executionTimeMs: 0.42
// }

โช Point-in-Time Recovery (v6)

Create snapshots and restore to any previous state:

const snap = await db.createSnapshot('before-migration');

// ... make changes ...
await db.set('config.version', '7.0.0');

// Roll back!
await db.restoreSnapshot(snap);

๐ŸŒ Pluggable Storage Adapters (v6)

Decouple the database from Node.js fs for browser/edge/serverless environments:

import { MemoryAdapter, FileSystemAdapter } from 'jsondb-high/adapters';

// In-memory (testing, ephemeral)
const mem = new MemoryAdapter();
await mem.write('{"test": 42}');
const data = await mem.read();

// Also: LocalStorageAdapter, IndexedDBAdapter, HttpAdapter

๐Ÿ“ก WAL Replication (v6)

Stream WAL entries to replicas with CRC32 integrity verification. Supports async and sync acknowledgment modes.

// WAL entries are automatically persisted
const db = new JSONDatabase('db.json', {
  durability: 'batched',
  walFlushMs: 5
});

await db.set('key', 'value');
await db.sync();

const status = db.walStatus();
// { enabled: true, committed_lsn: 42 }

๐Ÿชถ Lite Mode (v6)

Stripped-down WASM-only build (<2MB) for AWS Lambda, edge functions, and browsers:

# Build the lite distribution
bun run build:lite

# Output: ./lite/ with WASM binary + shim + adapters only
# No native .node files, no rayon parallelism
# Perfect for size-constrained environments

๐Ÿ“ฆ WASM Fallback (v6)

100% installation success โ€” the package automatically falls back to WASM when no native binary is available and Rust is not installed:

// Loading priority:
// 1. Prebuild .node binary (prebuilds/index.linux-x64-gnu.node)
// 2. Build from source via cargo (if Rust installed)
// 3. WASM fallback (always works, slightly slower)

// You'll see this warning in WASM mode:
// [jsondb-high] Running in WASM fallback mode.
// Performance will be reduced (no parallelism).

TypeScript Interfaces

interface DBOptions {
  indices?: IndexConfig[];
  wal?: boolean;
  encryptionKey?: string;
  autoSaveInterval?: number;
  lockMode?: 'exclusive' | 'shared' | 'none';
  lockTimeoutMs?: number;
  durability?: 'none' | 'lazy' | 'batched' | 'sync';
  walBatchSize?: number;
  walFlushMs?: number;
  schemas?: Record<string, Schema>;
  slowQueryThresholdMs?: number;
}

interface IndexConfig {
  name: string;
  path: string;
  field: string;
}

Schema Types

interface Schema {
  type: 'string' | 'number' | 'boolean' | 'array' | 'object';
  properties?: Record<string, Schema>;
  required?: string[];
  items?: Schema;
  enum?: any[];
  minimum?: number;
  maximum?: number;
  minLength?: number;
  maxLength?: number;
  pattern?: string;
  minItems?: number;
  maxItems?: number;
  uniqueItems?: boolean;
}

Query Types

interface ParallelFilter {
  field: string;
  op: 'eq' | 'ne' | 'gt' | 'gte' | 'lt' | 'lte' | 
      'contains' | 'startswith' | 'endswith' | 
      'in' | 'notin' | 'regex';
  value: any;
}

interface PaginateResult<T> {
  data: T[];
  meta: {
    total: number;
    pages: number;
    page: number;
    limit: number;
    hasNext: boolean;
    hasPrev: boolean;
  };
}