> Documentation index: https://pmdata.dev/llms.txt. Full docs in one file: https://pmdata.dev/llms-full.txt

# Overview

Source: https://pmdata.dev/docs/historical-data-details/overview

> Historical market data details for each supported exchange — available data type, date ranges

Historical data details describes data collection specifics for each supported data source and what's available via the PMData [HTTP API](https://pmdata.dev/docs/api-reference). Data is provided as Parquet files, with daily ZIP downloads available for supported datasets.

You'll find here details about:

- data quality — measures to prevent data loss and preserve latency characteristics similar to live trading
- historical data availability — since when historical data has been collected and is available
- captured real-time market data channels — historical market data is sourced from WebSocket real-time APIs and can be downloaded by data type. For example, to get historical trades of polymarket, channel trades needs to be provided alongside the requested `market_slug` via the [HTTP API](https://pmdata.dev/docs/api-reference).
- incidents — known PMData data issues where data may be missing or affected, with fixes documented in [product updates](https://pmdata.dev/docs/product-updates)

Our API treats data type and market or instrument identifier as separate inputs. For example, to download order book data, provide `l2` as the data type and the requested `market_slug`. If a channel offers a choice of update frequencies (e.g. 100ms updates vs. raw tick-by-tick data), the higher-frequency feed is always chosen and recorded.

## Market data collection overview

- Market data collection is performed in AWS `eu-west-2`, London, UK.
- When an exchange provides a choice of real-time data frequency for specific data types (e.g. order book data), the most granular, non-aggregated data feed is always collected.
- Each market uses two WebSocket connections in separate processes to help prevent data loss.
- At 00:00 UTC every day, a `book_snapshot` is generated from the previous day's L2 data to provide an [initial order book snapshot](#collected-order-book-data-details). Its `local_timestamp` and `timestamp` are both set to 00:00 UTC of that day.
- Each received message is timestamped at arrival time and stored in UTC as `timestamp[us]`.
- Timestamps come from a clock synchronized with chrony to [AWS PHC](https://aws.amazon.com/blogs/compute/its-about-time-microsecond-accurate-clocks-on-amazon-ec2-instances/), which AWS documents with a clock error bound typically under 40 µs. The bound is the most the clock can be off; the actual offset is usually far smaller. For reference, Google Cloud publishes no error bound for its standard NTP, and its own [chrony examples](https://docs.cloud.google.com/compute/docs/instances/configure-ntp) show ±257 µs and ±396 µs; Tardis.dev, for example, syncs its clock over [Google NTP](https://docs.tardis.dev/faq/data). Accurate timestamps keep modeled latency realistic and let you align PMData with your own Binance data or trading logs.
- Messages provided by exchanges' WebSocket feeds are stored without any modifications.
- Checks for new trading events(markets) available on an exchange are performed every hour.
- Market data collection services are constantly monitored both manually and via automated tools, including monitoring and alert notifications, and have built-in self-healing capabilities.
- There are multiple built-in checks that detect whether a connection to an exchange is healthy during data collection:
  - **Connection silence detection** — if no messages or control frames are received for more than 30 seconds, the connection is automatically restarted.
  - **Market data silence detection** — even if PONG messages are still received, the connection is automatically restarted if market data for the current subscriptions remains absent.
  - **Message format validation** — JSON format and data items are checked; errors are recorded and processing continues so that an invalid message does not interrupt collection.
- Known data issues and fixes are published in [product updates](https://pmdata.dev/docs/product-updates).

## Collected order book data details

Historical market data available via the [HTTP API](https://pmdata.dev/docs/api-reference) provides a `book_snapshot` at the beginning of each day (00:00 UTC), generated from the previous day's L2 data. Both `local_timestamp` and `timestamp` are set to 00:00 UTC of that day. If a WebSocket connection closes unexpectedly during collection, it is restarted and a new snapshot is provided through the fresh connection.

To be sure to receive the initial order book snapshot, historical data must be replayed from 00:00 UTC of that day. We do not deliberately disconnect to obtain daily snapshots and use two WebSocket connections for redundancy, so there is no **gap in historical data caused by scheduled re-subscription**.

## Per-exchange historical data details

Click any data source below to see its historical data details — available instruments, captured real-time channels, API access details and market data collection specifics.

- [Polymarket](https://pmdata.dev/docs/historical-data-details/polymarket)
- [Chainlink](https://pmdata.dev/docs/historical-data-details/chainlink)