Building a Universal Time-Travel Engine for Real-Time Event Streams

Visualizing real-time systems — order books, logs, infrastructure events — is still weirdly hard. Most tools are tied to a single exchange or vendor, built around JSON + slow backends, non-deterministic under load, and not reusable as components. I wanted something that could handle any event stream, reconstruct exact state at any point in time, and work as a composable primitive.

The Problem

Most time-travel visualization tools are:

  • Tied to a single exchange or vendor
  • Built around JSON + slow backends
  • Non-deterministic under load
  • Not reusable as components
  • Built for dashboards, not replayable state

I needed a solution that could:

  • Handle any event stream (order books, logs, Kafka, metrics)
  • Reconstruct exact state at any point in time
  • Work as a composable primitive
  • Provide deterministic replay

The Solution

Matchstick Time-Travel Engine (TTE) - A universal, high-performance, deterministic time-travel engine for real-time event streams.

Core Concept

  • Snapshot + diff-based state reconstruction
  • Deterministic time travel (jump, scrub, replay)
  • Pluggable adapters for any data source
  • Pluggable renderers for any visualization

Architecture

Data Source (Kraken, Binance, logs, Kafka)
    ↓
Adapter (converts to canonical DiffEvent<T>)
    ↓
TimeTravelEngine (snapshots + diff log)
    ↓
Renderer (React, Canvas, WebGL, etc.)

Technical Highlights

Deterministic State Reconstruction

  • Maintains snapshots at configurable intervals (default: 1 second)
  • Stores diffs in a circular buffer
  • Can reconstruct exact state at any timestamp
  • <10ms event-to-render latency

Time-Travel Controls

  • jumpTo(timestamp) - Jump to specific point in time
  • seek(offsetMs) - Move forward/backward by time offset
  • replay(speed) - Replay at 1x, 2x, 10x speed
  • pause() / resume() / setLive() - Control playback

Pluggable Architecture

  • Adapters: Convert any data source to canonical DiffEvent<T> format
  • Renderers: Visualize reconstructed state (React components, Canvas, WebGL)
  • Framework Agnostic: Core engine is pure TypeScript, works with any UI

Example Usage

import { TimeTravelEngine } from '@matchstick/tte-core';
import { KrakenAdapter, orderBookReducer } from '@matchstick/adapter-kraken';
import { OrderBookViewer } from '@matchstick/orderbook-viewer';

// Initialize engine with order book state
const engine = new TimeTravelEngine(
  initialState,
  orderBookReducer,
  { snapshotInterval: 1000 }
);

// Connect Kraken adapter
const adapter = new KrakenAdapter({ pairs: ['XBT/USD'], depth: 10 });
adapter.onDiff((diff) => engine.ingest(diff));
await adapter.connect();

// Use React component
<OrderBookViewer engine={engine} maxLevels={10} />

// Control time travel
engine.pause();           // Pause live updates
engine.jumpTo(timestamp); // Jump to specific time
engine.replay(10);        // Replay at 10x speed
engine.setLive();         // Return to live mode

Impact

Beyond Order Books

  • SRE: Log adapter + list renderer for debugging
  • Infrastructure: Kafka adapter + partition renderer for monitoring
  • Trading: Order book adapter + depth chart renderer
  • Any Time-Series: Pluggable adapters + custom renderers

Performance

  • <10ms event-to-render latency
  • 60fps rendering
  • Handles high-frequency event streams
  • Memory efficient with snapshot + diff model

What Makes It Different

Most time-travel tools are built for specific use cases (order books OR logs OR metrics), tied to specific vendors or platforms, and non-deterministic (can't guarantee exact state reconstruction).

TTE is:

  • Universal - Works with any event stream via adapters
  • Deterministic - Exact state reconstruction at any timestamp
  • Composable - Small primitive that works with any UI framework
  • High Performance - <10ms latency, 60fps rendering

Tech Stack

  • Core Engine: TypeScript (pure, framework-agnostic)
  • Adapters: Kraken WebSocket (Binance, Coinbase coming soon)
  • Renderers: React components (Canvas/WebGL planned)
  • State Management: Snapshot + diff log with circular buffer
  • Performance: Optimized for high-frequency event streams

Current Status

v0.1.0 Released:

  • ✅ Core time-travel engine
  • ✅ Kraken WebSocket adapter
  • ✅ React order book viewer
  • ✅ Live demo example
  • ✅ Comprehensive documentation

Roadmap:

  • Binance and Coinbase adapters
  • Canvas/WebGL renderer for high-performance visualization
  • IndexedDB persistence for long replay sessions
  • WASM acceleration for diff merging
  • Time-travel replay demo with scrubbing UI

What I Learned

Snapshot + diff model is incredibly powerful for time-travel. Deterministic state reconstruction requires careful design, and pluggable architecture enables reuse across domains. Performance matters - <10ms latency requires optimization. Small, composable primitives are more valuable than monolithic tools.

Links

Perfect for visualizing any real-time event stream where you need to debug what happened at a specific time, replay events at different speeds, jump to specific timestamps, or understand state changes over time.