Overview

You'll find here details about:

• data quality — measures to prevent data loss and preserve latency characteristics similar to live trading
• historical data availability — since when historical data has been collected and is available
• captured real-time market data channels — historical market data is sourced from WebSocket real-time APIs and can be downloaded by data type. For example, to get historical trades of polymarket, channel trades needs to be provided alongside the requested market_slug via the HTTP API.
• incidents — known PMData data issues where data may be missing or affected, with fixes documented in product updates

Our API treats data type and market or instrument identifier as separate inputs. For example, to download order book data, provide l2 as the data type and the requested market_slug. If a channel offers a choice of update frequencies (e.g. 100ms updates vs. raw tick-by-tick data), the higher-frequency feed is always chosen and recorded.

Market data collection overview

• Market data collection is performed in AWS eu-west-2, London, UK.
• When an exchange provides a choice of real-time data frequency for specific data types (e.g. order book data), the most granular, non-aggregated data feed is always collected.
• Each market uses two WebSocket connections in separate processes to help prevent data loss.
• At 00:00 UTC every day, a book_snapshot is generated from the previous day's L2 data to provide an initial order book snapshot. Its local_timestamp and timestamp are both set to 00:00 UTC of that day.
• Each received message is timestamped at arrival time and stored in UTC as timestamp[us].
• Timestamps come from a clock synchronized with chrony to AWS PHC, which AWS documents with a clock error bound typically under 40 µs. The bound is the most the clock can be off; the actual offset is usually far smaller. For reference, Google Cloud publishes no error bound for its standard NTP, and its own chrony examples show ±257 µs and ±396 µs; Tardis.dev, for example, syncs its clock over Google NTP. Accurate timestamps keep modeled latency realistic and let you align PMData with your own Binance data or trading logs.
• Messages provided by exchanges' WebSocket feeds are stored without any modifications.
• Checks for new trading events(markets) available on an exchange are performed every hour.
• Market data collection services are constantly monitored both manually and via automated tools, including monitoring and alert notifications, and have built-in self-healing capabilities.
• There are multiple built-in checks that detect whether a connection to an exchange is healthy during data collection:
  • Connection silence detection — if no messages or control frames are received for more than 30 seconds, the connection is automatically restarted.
  • Market data silence detection — even if PONG messages are still received, the connection is automatically restarted if market data for the current subscriptions remains absent.
  • Message format validation — JSON format and data items are checked; errors are recorded and processing continues so that an invalid message does not interrupt collection.
• Known data issues and fixes are published in product updates.

Collected order book data details

Historical market data available via the HTTP API provides a book_snapshot at the beginning of each day (00:00 UTC), generated from the previous day's L2 data. Both local_timestamp and timestamp are set to 00:00 UTC of that day. If a WebSocket connection closes unexpectedly during collection, it is restarted and a new snapshot is provided through the fresh connection.

To be sure to receive the initial order book snapshot, historical data must be replayed from 00:00 UTC of that day. We do not deliberately disconnect to obtain daily snapshots and use two WebSocket connections for redundancy, so there is no gap in historical data caused by scheduled re-subscription.

Per-exchange historical data details

Click any data source below to see its historical data details — available instruments, captured real-time channels, API access details and market data collection specifics.

• Polymarket
• Chainlink