Documentation

Secure, indexed, single-file JSON database for Node.js. AES-256-GCM encryption, atomic writes, indexes, schema validation, transactions, batch ops, and middleware — all in a simple async API.

Install

npm install json-database-st lodash

Requires Node.js ≥ 14.

Quickstart

// CommonJS
const JSONDatabase = require('json-database-st');
const path = require('path');

const db = new JSONDatabase(path.join(__dirname, 'data.json'), {
  // Generate once and store securely (64 hex chars → 32 bytes)
  // encryptionKey: require('crypto').randomBytes(32).toString('hex'),
  indices: [ { name: 'user-email', path: 'users', field: 'email', unique: true } ]
});

await db.set('users.alice', { email: 'alice@example.com', name: 'Alice' });
const alice = await db.findByIndex('user-email', 'alice@example.com');
console.log(alice);

await db.close();

ESM usage

// In ESM modules, require via createRequire
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
const JSONDatabase = require('json-database-st');

Constructor & Options

new JSONDatabase(filename, {
  encryptionKey: undefined,            // string | undefined — 64 hex chars (32 bytes)
  prettyPrint: false,                   // bool — pretty JSON when not encrypted
  writeOnChange: true,                  // bool — skip disk write if unchanged
  schema: null,                         // object with safeParse(data)
  indices: []                           // [{ name, path, field, unique? }]
})
  • encryptionKey: 64‑char hex → AES‑256‑GCM at rest. Wrong key or tamper → secure failure.
  • schema: Provide a Zod/Joi‑like object with safeParse. Rejects writes if invalid.
  • indices: Define fast lookups via findByIndex(). Optional unique constraint.

API

// read
await db.get(path?, defaultValue?)           // path optional → full cache
await db.has(path)                           // boolean
await db.find(collectionPath, predicate)     // lodash predicate
await db.findByIndex(indexName, value)       // O(1) lookup

// write
await db.set(path, value)
await db.delete(path)                        // returns true even if path missing
await db.push(path, ...items)                // unique by deep equality
await db.pull(path, ...itemsToRemove)        // deep-equality removal

// atomic workflows
await db.transaction(async (data) => {       // must return data
  data.count = (data.count||0) + 1; return data;
})

await db.batch([
  { type:'set', path:'users.alice', value:{ name:'Alice' } },
  { type:'push', path:'tags', values:['a','b'] },
  { type:'delete', path:'old.key' }
], { stopOnError:false })

// lifecycle
db.getStats()                                // { reads,writes,cacheHits }
await db.close()

Method reference

  • get(path?, defaultValue?): Returns value at path or entire cache; never writes.
  • set(path, value): Writes value; validates schema; updates indices; emits write/change.
  • has(path): Boolean presence check.
  • delete(path): Removes path; resolves to true even if already missing.
  • push(path, ...items): Creates array if missing; deep-unique append.
  • pull(path, ...items): Deep-equality filter removal.
  • find(collectionPath, predicate): Lodash-style predicate over object/array.
  • findByIndex(indexName, value): O(1) lookup into pre-built index.
  • transaction(fn): Single atomic rewrite; returning undefined aborts.
  • batch(ops, { stopOnError }): Executes ops in one rewrite; logs errors unless stopOnError.
  • getStats(): Shallow copy of counters.
  • close(): Flushes outstanding writes, clears cache and indices.

Security

  • Encryption at rest: AES‑256‑GCM with random IV and auth tag. Tamper or wrong key → secure failure.
  • Key format: Provide 64‑char hex string (32 bytes). Store in environment variables.
  • Path traversal protection: DB file must resolve under process.cwd().
  • Crash recovery: Atomic write via .tmp + rename; on boot, recovers if temp file exists.
// Generate a secure key once
const key = require('crypto').randomBytes(32).toString('hex');
console.log(key);

Schema validation

Pass any object exposing safeParse(data). On failure, throws ValidationError and aborts the write.

const { z } = require('zod');
const schema = z.object({ users: z.record(z.object({ email: z.string().email(), name: z.string() })) });
const db = new JSONDatabase('db.json', { schema });

await db.set('users.alice', { email: 'alice@example.com', name: 'Alice' });

Indexing

const db = new JSONDatabase('db.json', {
  indices: [ { name:'user-email', path:'users', field:'email', unique:true } ]
});

await db.set('users.alice', { email:'alice@example.com', name:'Alice' });
await db.set('users.bob',   { email:'bob@example.com',   name:'Bob'   });

const user = await db.findByIndex('user-email', 'bob@example.com');
// → { email:'bob@example.com', name:'Bob' }

Unique violations throw IndexViolationError.

  • Indexes map field values to object keys under path for O(1) reads.
  • Indices rebuild on init and update incrementally on writes.
  • Define multiple indices by adding more entries to indices.

Transactions

await db.transaction((data) => {
  data.balance = (data.balance || 0) + 100;
  return data; // returning undefined aborts the write
});
  • Receives a deep clone of current data; must return the new data object.
  • All validations and index updates happen once per transaction.

Batch operations

await db.batch([
  { type:'set', path:'accounts.a', value:100 },
  { type:'set', path:'accounts.b', value:200 },
  { type:'push', path:'log', values:['boot'] }
], { stopOnError: false });
  • Runs all operations in-memory, then performs a single atomic write.
  • stopOnError: when true, throws on first invalid op; when false, logs errors and continues.

Array helpers

await db.set('tags', ['a','b']);
await db.push('tags', 'b', 'c');     // → ['a','b','c'] (unique)
await db.pull('tags', 'a');          // → ['b','c']

Middleware

Register hooks on before/after for operations set|delete|push|pull|transaction|batch.

db.before('set', 'users.*', (ctx) => {      // wildcard match
  ctx.value.updatedAt = Date.now();
  return ctx;                                // must return a context object
});

db.after('set', 'users.*', ({ path, value, finalData }) => {
  console.log('wrote', path, value);
});
  • before must return a context object; returning falsy aborts the operation.
  • Patterns use * to match a single path segment: users.*.name.

Events

db.on('write',  (e) => console.log('write', e));     // { filename, timestamp }
db.on('change', (e) => console.log('change', e));    // { oldValue, newValue }
db.on('error',  (e) => console.error('db error', e));

Error types

  • DBInitializationError — file parse/init failures
  • TransactionError — transaction fn returned undefined
  • ValidationError — schema.safeParse failed
  • IndexViolationError — unique index violated
  • SecurityError — path traversal or bad encryption key
try {
  await db.set('users.dupe', { email: 'alice@example.com' });
} catch (err) {
  if (err.name === 'IndexViolationError') {
    // handle unique violation
  }
}

Concurrency & Locking

Single Node.js process per DB file is recommended. Writes use an OS‑level advisory lock via proper-lockfile to serialize access and an atomic .tmp + rename strategy.

  • Parallel writes in one process are queued.
  • Multi‑process writes to the same file are not supported.

Tips & Limits

  • Loads entire file into memory on init — size accordingly.
  • Optimized for single-process usage per DB file.
  • Batch updates to reduce rewrite cost on large files.
  • Keep hot lookups in indexes; avoid wide blobs for per-request reads.

FAQ

How do I rotate the encryption key?

Create a new instance pointing to a new file with the new key, read all data from the old DB, write to the new one, then swap files.

Can I define multiple indices?

Yes. Provide multiple entries in indices. Each one is maintained independently.

Does get() hit the disk?

No. Data is cached in memory after initialization.