Deepkit v2

[marcj]

Deepkit v2

Performance at a Glance

	vs v1	vs bson-js 7.x
BSON serialize	3–63x faster	2–224x
BSON deserialize	1.8–7x faster	2.5–13x
Type serialize	1.5–14x faster	—
Type deserialize	1.9–7x faster	—
Type validate	3–17x faster	—
Type is()	11–18x faster	—

25+ bug fixes, 8 new features, 2,801 tests migrated to node:test

---

Summary

Deepkit v2 is a ground-up rewrite of the framework's core JIT, serialization, and BSON layers — delivering order-of-magnitude performance improvements while adding CSP compliance, security hardening, and a cleaner public API.

1. JIT Architecture Rewrite — New expression-tree based jit.ts in @deepkit/core. Zero new Function() calls, CSP-compliant, tiered execution (interpret first, JIT-compile hot paths).

2. BSON Package Rewrite — Complete rewrite of @deepkit/bson with shape-learning JIT deserializer, zero-copy serialization, and security hardening. 3–63x faster than v1, 2–224x faster than bson-js 7.x.

3. Type Serializer Rewrite — @deepkit/type serializer migrated to jit.fn() API. Union validation 1000x faster via O(1) discriminator dispatch. ReceiveType overhead reduced 90%. All 12 benchmarks beat v1.

Fixes: #241, #285, #318, #352, #356, #439, #458, #478, #505, #508, #509, #524, #555, #562, #573, #589, #590, #598, #601, #612, #614, #634, #636, #653, #664, #668, #676

Features: #395, #419, #441, #456, #577, #582, #600

---

Breaking Changes

1. feat(type,bson)!: Type-driven Reference serialization with Inline annotation

& Reference now always serializes as FK (primary key only), regardless of runtime object state. Previously, serialization depended on isReferenceInstance() which was unpredictable.

New API:

class Post {
    // Always serializes as FK (e.g., { author: 2 })
    author: User & Reference;

    // Serializes as nested object for JSON and RPC (throws if not loaded).
    // ORM/MongoDB/BSON always stores as FK regardless of Inline.
    editor: User & Reference & Inline;

    // Inline only for JSON (not RPC BSON)
    reviewer: User & Reference & Inline<{ only: ['json'] }>;
}

Inline controls serialization for JSON and RPC BSON output. MongoDB communication always serializes references as FK — Inline has no effect on MongoDB storage.

Migration: If your code relied on joinWith() affecting serialization output, use & Reference & Inline on the type definition instead.

2. feat(bson)!: BSON API overhaul

Removed exports:

• Writer class
• BaseParser class
• BSONBinarySerializer class
• ValueWithBSONSerializer type
• AutoBuffer utility
• getBSONSizer() function
• stringByteLength() utility
• getBsonEncoder() (renamed to getBSONEncoder())

Changed return types:

• getBSONSerializer<T>() now returns (data: T) => [Uint8Array, number] (buffer + size tuple, zero-copy)
• Was: (data: T) => Uint8Array

New exports:

• getBSONEncoder<T>() — high-level encode/decode pair
• deserializeBSONWithoutOptimiser() — public slow-path deserializer
• SerializeResult type alias for [Uint8Array, number]

3. feat(core)!: New jit.ts — expression tree JIT/Exec architecture

The old CompilerContext-based JIT in @deepkit/core is replaced with a new Builder API (jit.ts). This is an internal API used by @deepkit/type and @deepkit/bson — not directly user-facing, but affects anyone extending the serializer.

Key changes:

• CompilerContext → Builder with expression tree model
• new Function() → closure-based executors (CSP-compatible)
• Supports tiered execution: interpret first, compile to JIT after N calls

---

New Features

@deepkit/type

Feature	Issue	Description
NanoId type support	#419	Native NanoId type with validation and serialization
Union constraint errors	#577	Show specific field-level errors for union validation failures
isStrict<T>()	—	Strict type guard without coercion
isWeak<T>()	—	Maximum-performance type guard (minimal checks)
Inline annotation	—	Control Reference serialization (see Breaking Changes)

@deepkit/http

Feature	Issue	Description
Express-compatible methods	#285	req.get() and req.header() for Express middleware compat
Built-in CORS support	#441	Native CORS middleware with configurable origins/methods/headers

@deepkit/type-compiler

Feature	Issue	Description
Improved DeepkitLoader API	#456	Better bundler integration for receiving types across files
tsconfig extends as array	#600	Support array syntax in tsconfig extends field

@deepkit/filesystem-aws-s3

Feature	Description
forcePathStyle option	Support forcePathStyle for S3-compatible services (MinIO, etc.)

@deepkit/framework

Feature	Issue	Description
Custom CRUD identifiers	#395	Support custom identifier fields in auto-generated CRUD routes
Replace faker dependency	#582	Replaced deprecated faker with @faker-js/faker

@deepkit/core

Feature	Description
Error code system	DeepkitError base class with DK-T###, DK-B###, etc. codes
Builder API extensions	forRange, forOf, arithmetic/bitwise ops, throw_, cond, concat

@deepkit/bson

Feature	Description
Shape-learning JIT deserializer	Learns document shapes at runtime, generates specialized JIT code per shape
Circular reference detection	Depth-based extraction prevents infinite loops
Security hardening	Prototype pollution protection, bounds checking, size validation
BinaryBigInt support	Native BSON binary representation for BigInt values

---

Bug Fixes

@deepkit/type-compiler

Fix	Issue	Description
External types produce broken output	#352	Emit 'any' bytecode for external types instead of invalid JS
Windows backslash path delimiters	#356	Escape backslashes in Windows file paths
Optional chaining SyntaxError	#612	Resolve SyntaxError when optional chaining meets type arguments
Function type hoisting	#664	Hoist function __types declarations
Named re-exports missing types	#634	Auto re-export __Ω symbols with named re-exports
Exclude declare statements	#601	Don't emit type info for declare statements
External import types	#555	Graceful degradation for external library types
InferType resolution	#509	Replace entire TypeReferenceNode for infer types
Index.ts re-export failures	#318	Fixed via #634 named re-export fix

@deepkit/type

Fix	Issue	Description
Circular import error	#562	Improve NoTypeReceived error messages for better DX
Conditional type inference	#524	Correct tuple inference with rest element before infer
Missing runtime type error	#508	Improve error messages with actionable guidance
Stack overflow on large unions	#478	Prevent stack overflow for large literal unions
Circular references in validation	#505	Handle circular references in ValidationErrorItem.toString()
SuperClass serialization	#241	Skip serializing superClass when parent has no __type
Custom Partial<T> shadow	—	Remove custom Partial<T> that shadowed TypeScript built-in
Optional chaining in JIT	—	Remove optional chaining from JIT-generated code
Constructor call exclusion	—	Exclude constructor calls from hasDefaultFunctionExpression

@deepkit/http

Fix	Issue	Description
Middleware error handling	#439	Improve error propagation with correct status codes
HttpBody in separate files	#458	Fix parameter injection across file boundaries
HttpHeader case sensitivity	#653	Make header parameter matching case-insensitive
Middleware response event	#590	Ensure onResponse fires when middleware ends response early
HttpError propagation	#589	Propagate HttpError from middleware with correct status code

@deepkit/orm

Fix	Issue	Description
Identity map hydration	#636	Upgrade reference proxies when same entity joined via different path
count() with pagination	#668	count() ignores pagination to return total count
withChangeDetection clone	—	Include withChangeDetection in DatabaseQueryModel.clone()
deleteResult.modified	—	Set deleteResult.modified in MemoryDatabaseAdapter

@deepkit/bson

Fix	Issue	Description
NaN serialization	#573	Serialize NaN as 0 instead of skipping
Union error messages	#676	Improve error messages for union type mismatches

@deepkit/rpc

Fix	Description
Subject premature GC	Prevent FinalizationRegistry from firing during active subscriptions (V8 marks variables as "dead" during await)
Subscribe type correctness	Fix TypeScript types for subscribe.apply arguments
Error messages	Improve error messages with controller/method context

Other

Fix	Package	Issue	Description
Missing shebang	sql, desktop-ui	#598	Add shebang to CLI bin files
HttpQuery validator leak	http	#614	Fix validator expression leak in HttpQuery
Error classes	type, bson	—	Replace plain Error with proper DeepkitError/BSONError subclasses

---

Performance Improvements

@deepkit/type

Optimization	Before	After	Improvement
Union validation (discriminated)	~5K ops/sec	~5M ops/sec	1000x
deserialize<T>() with ReceiveType	1M ops/sec	3.6M ops/sec	3.6x
validate<T>() with ReceiveType	1M ops/sec	5.5M ops/sec	5.5x
is<T>() with ReceiveType	1M ops/sec	7.4M ops/sec	7.4x
Nullish checks in guards	—	—	x==null instead of x===undefined||x===null
Serialize (no groups)	—	—	Direct object literal, skip wrapper

ReceiveType overhead breakdown (was 325ns → now ~32ns):

• Eliminated Ω array wrapper (57ns)
• Singleton NamingStrategy (101ns allocation)
• Hybrid direct argument passing eliminates Ω side-channel entirely

Union validation fix: Added detectDiscriminator() to guardUnionFast — uses O(1) switch dispatch instead of linear member iteration. Error collection via validateDiscriminatedUnionWithErrors runtime.

Index signature optimization: Inline index signature validation/serialization directly in b.forIn loops using state.forKey(key).build(). Pre-build serializer/deserializer/type-guard functions at JIT time instead of rebuilding per-key per-call. Cache extracted nested functions in buildExtractedCall.

vs v1 performance (pre-resolved fn() API):

Benchmark	v2	v1	Change
Small model deserialize	41.7M	21.7M	+92%
Medium model deserialize	9.7M	4.3M	+126%
Union deserialize	16.6M	2.2M	+655%
Small model serialize	135.6M	19.0M	+614%
Medium model serialize	1.9M	1.3M	+46%
Union serialize	34.0M	2.4M	+1317%
Small model validate	64.8M	20.4M	+218%
Medium model validate	20.0M	4.9M	+308%
Union validate	8.4M	0.5M	+1580%
Small model is()	374.2M	21.1M	+1674%
Medium model is()	52.6M	4.9M	+973%
Union is()	8.7M	0.5M	+1640%

All 12 benchmarks beat v1. Fastest improvements: small is() +1674%, union serialize +1317%, union validate +1580%.

@deepkit/bson

Single document performance (pre-resolved API):

Benchmark	v2	v1	vs v1	vs bson-js 7.x
Serialize
int32 (1 field)	1,198M	19.1M	+6172%	203x
float64 (1 field)	1,043M	18.8M	+5448%	181x
string (1 field)	145M	11.7M	+1139%	28x
boolean (1 field)	1,208M	20.9M	+5681%	215x
MongoId (1 field)	40.1M	9.8M	+309%	10x
UUID (1 field)	24.9M	6.2M	+302%	6x
int32 x3	667M	12.6M	+5194%	224x
mixed x3	604M	11.7M	+5062%	206x
sensor (4 numeric)	342M	7.5M	+4460%	153x
user profile (6 fields)	16.8M	4.9M	+243%	10x
API meta (3 strings)	16.0M	4.9M	+227%	7x
minimal (_id only)	40.7M	8.9M	+357%	10x
Deserialize
int32 (1 field)	186M	28.5M	+553%	13x
float64 (1 field)	81.0M	27.4M	+196%	6x
string (1 field)	43.6M	14.7M	+197%	5x
boolean (1 field)	111M	30.0M	+270%	10x
MongoId (1 field)	27.1M	10.2M	+166%	5x
UUID (1 field)	17.6M	5.1M	+245%	5x
int32 x3	56.5M	21.1M	+168%	8x
mixed x3	44.8M	20.3M	+121%	6x
sensor (4 numeric)	23.3M	13.1M	+78%	10x
user profile (6 fields)	11.6M	—	—	7x
API meta (3 strings)	12.7M	3.7M	+243%	6x
minimal (_id only)	27.0M	10.0M	+170%	5x

All single-document benchmarks beat v1. Serialize 3–63x faster (buffer reuse + zero-copy tuple return), deserialize 1.8–7x faster (shape-learning JIT + tiered UTF-8 decoder + hexTable2 optimization). All benchmarks 5–224x faster than bson-js 7.x.

Array performance (MongoDB cursor response { cursor: { firstBatch: T[] } }):

Benchmark	Serialize	vs bson-js 7.x	Deserialize	vs bson-js 7.x
sensor[] 10 items	8.7M ops/sec	39x	1.7M ops/sec	7.5x
sensor[] 1K items	100K ops/sec	37x	17K ops/sec	6.8x
mixed[] 10 items	1.9M ops/sec	12x	1.0M ops/sec	5.6x
mixed[] 1K items	20K ops/sec	12x	10K ops/sec	5.6x
string[] 100 items	700K ops/sec	2x	500K ops/sec	2.5x
number[] 100 items	4.2M ops/sec	10x	1.5M ops/sec	2.5x

Array performance tested in the standard MongoDB response pattern at depth 3 — the most common real-world access pattern. Deepkit maintains 2–39x advantage over bson-js 7.x at all array sizes and element types.

Shape-learning JIT: Documents are profiled at runtime. After learning the shape (field order, types), a specialized JIT reader is generated that skips field name parsing. Falls back to interpreted path for unknown shapes. Provides 21x improvement for union types.

hexTable2 optimization: 65,536-entry lookup table maps byte-pairs to 4-char hex strings. Eliminates per-byte hex conversion for UUID/MongoId.

@deepkit/http

Optimization	Description
Pre-compiled middleware resolvers	Cache resolver compilation, avoid repeated compilation per request

---

Test Migration: Jest → node:test

Migrated three core packages from Jest to node:test with a shared @deepkit/run/expect assertion shim.

Package	Tests	Status
@deepkit/bson	549	All pass
@deepkit/type-compiler	277	All pass
@deepkit/type	1,975	All pass (4 pre-existing skips)
Total	2,801	All pass

Changes:

• Created packages/run/expect.ts — shared expect() shim with matchers: toBe, toEqual, toStrictEqual, toBeInstanceOf, toContain, toMatch, toThrow, toBeGreaterThan, toBeTruthy, toBeFalsy, toBeNull, toBeUndefined, toBeDefined, toHaveProperty, toHaveLength, toMatchObject, resolves, rejects
• Added test:node scripts to package.json files
• Import pattern: import { expect } from '@deepkit/run/expect'
• Added useDefineForClassFields: false to tsconfig.base.json (ES2022 defaults it to true, which changes class field initialization semantics)

---

Infrastructure

Benchmark Suite (new)

Comprehensive benchmark infrastructure at benchmarks/:

• 306 benchmarks across 15 suites
• Core benchmarks: serialization, validation, BSON, ORM, HTTP, RPC, injector, change-detection
• Comparison benchmarks: vs Zod, vs class-transformer, vs bson-js, vs Typia
• V8 pattern microbenchmarks: function creation, property access, nullish checks, etc.
• Pre-refactor baseline saved for regression detection
• JSON, Markdown, SVG report generation
• npm run benchmark -- --compare-baseline fails on >20% regression

Docker Compose Test Environment

Full test stack at docker-compose.yml:

• PostgreSQL (15432), MySQL (13306), MongoDB replica set (27117), Redis (16379)
• SFTP, FTP, MinIO/S3, Fake GCS for filesystem adapter tests
• Alternative ports to avoid conflicts with local services

Error Code System

All packages now use DeepkitError base class with coded errors:

• DK-T### (@deepkit/type), DK-B### (@deepkit/bson), DK-O### (@deepkit/orm)
• DK-I### (@deepkit/injector), DK-H### (@deepkit/http), DK-R### (@deepkit/rpc)
• Each code links to documentation

Pre-commit Hooks (lefthook)

• Typecheck gate (npm run typecheck)
• Prettier formatting
• Conventional commit message enforcement

---

Open TODOs (Post-Merge)

CRITICAL: Consumer Package Migration

5 packages have broken imports due to the BSON API overhaul and need updating before the full framework compiles:

Package	Broken Imports	Complexity
@deepkit/rpc	Writer, getBSONSizer — protocol binary serialization	High — needs refactored message construction
@deepkit/mongo	BSONBinarySerializer, ValueWithBSONSerializer, Writer, getBSONSizer	High — needs new serializer composition pattern
@deepkit/broker	getBSONSerializer return type changed to [buffer, size] tuple	Low — destructure tuple at call sites
@deepkit/broker-redis	AutoBuffer removed, getBsonEncoder → getBSONEncoder case fix	Medium — remove AutoBuffer, fix case + decode() signature
@deepkit/framework-debug-api	Writer, BaseParser, getBSONSizer, stringByteLength	Medium — implement local binary I/O utilities

Detailed migration notes: See .claude/handover.md Suppressed Issues section.

RPC Protocol Rewrite

The RPC protocol (packages/rpc/src/protocol.ts) uses low-level binary operations (Writer, getBSONSizer) to construct message envelopes with precise size control. This needs either:

• A refactor to use the new getBSONSerializer() tuple API (serialize first, then build envelope)
• Or re-exporting Writer/sizer utilities from @deepkit/bson for low-level consumers

MongoDB Client Rewrite

The MongoDB client (packages/mongo/src/client/connection.ts) has similar needs — constructs MongoDB wire protocol messages using Writer and pre-calculated sizes. The MongoBinarySerializer class (extends removed BSONBinarySerializer) needs architectural replacement.

Additional Package Migrations to node:test

Remaining packages still use Jest:

• @deepkit/injector, @deepkit/orm, @deepkit/http, @deepkit/framework
• @deepkit/mysql, @deepkit/postgres, @deepkit/sqlite
• @deepkit/mongo, @deepkit/rpc

4 Skipped Tests

packages/type/tests/serializer.spec.ts has 4 test.skip (pre-existing from Jest):

• onLoad call (×3) — lines 1153, 1180, 1204
• extend with custom type — line 1363