Telemetry & Monitoring

Hydra implements a high-performance telemetry stack designed for sub-millisecond visibility into trading operations. It uses a push-based architecture to stream metrics to InfluxDB via Telegraf.

Why Push-Based?

In high-frequency trading (HFT), deterministic execution is critical. Traditional “pull-based” monitoring systems (like Prometheus) can cause performance issues:

1. CPU Jitter: The act of scraping metrics can cause CPU spikes that delay order execution.
2. Stop-the-world: Metric collection often happens during garbage collection or other blocking events, exactly when you need visibility the most.

Hydra’s push-based model sends metrics over non-blocking UDP (or HTTP), ensuring that the trading engine is never interrupted by monitoring.

Key Metrics

Event Loop Lag

The EventLoopMonitor detects when the JavaScript event loop is blocked. This is crucial for identifying GC pauses or poorly written synchronous code that could delay orders.

import { EventLoopMonitor } from "./core/metrics/eventLoopMonitor";

const monitor = new EventLoopMonitor(telemetryService);
monitor.start();

• Metric: hydra_event_loop
• Fields: lag_ms, elapsed_ms
• Threshold: Warnings are logged if lag exceeds 50ms.

Latency Percentiles

The LatencyTracker provides high-precision tracking of event processing times.

import { LatencyTracker } from "./core/metrics/latencyTracker";

const tracker = new LatencyTracker();
tracker.record("tick_to_trade", 1.25); // Record 1.25ms latency

const stats = tracker.getStats("tick_to_trade");
console.log(`P99 Latency: ${stats.p99}ms`);

• Measurements: P50, P90, P99, P99.9, Min, Max, Mean.

Tick-to-Trade Latency

For every reference price update, Hydra tracks:

1. Network Latency: arrivalTsMs - exchangeTsMs (Time from Binance to Hydra)
2. Processing Latency: sendTsMs - arrivalTsMs (Time from data arrival to order send)

Configuration

Telemetry is configured in config.yaml:

telemetry:
  enabled: true
  host: "127.0.0.1"
  port: 8094
  protocol: udp        # 'udp' for low-latency, 'http' for reliability
  database: "hydra"
  batchSize: 100
  flushIntervalMs: 100 # High frequency flushing for real-time dashboards

Dashboard Integration

We recommend using Grafana with an InfluxDB data source to visualize these metrics. Example measurements:

Measurement	Description	Key Fields
hydra_orderbook	Polymarket book state	up_bid, up_ask, up_spread
hydra_reference_price	Binance price updates	price, network_latency_ms
hydra_signal	Strategy signals	expected_edge, expected_profit_rate
hydra_risk_trip	Safety breaker events	breaker_type
hydra_event_loop	Event loop health	lag_ms

Local Setup with Docker

The easiest way to run the telemetry stack locally is via Docker:

# Start InfluxDB and Telegraf
docker-compose up -d influxdb telegraf

Ensure your telegraf.conf is configured to listen on the UDP port specified in your Hydra config.