Metrics reference

Metrics contains counters, gauges, and histogram snapshots. A counter is a non-negative integer, a gauge represents current state, and a histogram records a distribution of observed values. The scan and prefetch-reader metrics described below are returned as point-in-time snapshots. Modifying a returned snapshot does not modify the component that produced it.

Scan planning

The names below are declared by ScanMetrics. last* counters are replaced after each successful CreatePlan() call. Histograms and the cache hit/miss counters accumulate for the lifetime of the scan. They are unrelated to SetReadSchema() and ReadAheadCache::Reset(). The first six names match Java ScanMetrics; the remaining names are C++-only.

Name	Type	Unit	Meaning
lastScanDuration	counter	milliseconds	Duration of the last successful plan
scanDuration	histogram	milliseconds	Distribution of successful plan durations
lastScannedSnapshotId	counter	snapshot ID	Snapshot used by the last plan, or 0
lastScannedManifests	counter	files	Manifest files selected by the last plan
lastScanSkippedTableFiles	counter	files	Table files skipped by the last plan
lastScanResultedTableFiles	counter	files	Table files returned by the last plan
lastManifestReadDuration	counter	milliseconds	Manifest-list and entry read time for the last plan
manifestReadDuration	histogram	milliseconds	Distribution of manifest read times
lastSnapshotCacheEnabled	counter	boolean	Whether snapshot manifest-entry cache was eligible
lastSnapshotCacheHit	counter	boolean	Whether the last eligible lookup hit
snapshotCacheHits	counter	lookups	Cumulative exact-snapshot cache hits
snapshotCacheMisses	counter	lookups	Cumulative eligible cache misses
lastSnapshotCacheLoadDuration	counter	milliseconds	Cache load time for the last plan
snapshotCacheLoadDuration	histogram	milliseconds	Distribution of cache load times
lastSnapshotCacheStoreDuration	counter	milliseconds	Cache store time for the last plan; 0 when not stored
snapshotCacheStoreDuration	histogram	milliseconds	Distribution of cache store times
lastLazyDecodeScannedRows	counter	manifest rows	Candidate manifest rows inspected by the last plan
lastLazyDecodeMaterializedRows	counter	manifest rows	Manifest rows retained after lazy filtering

Prefetch reader

The names below are declared by PrefetchMetrics. Counters and histograms accumulate for the lifetime of the prefetch reader, including across SetReadSchema(). enabled and parallelism describe the most recently initialized schema. queue-depth is reset by SetReadSchema() and Close(); queue-depth.max remains the lifetime maximum. These metrics are C++-only and have no counterparts in Java Paimon.

Name	Type	Unit	Meaning
prefetch.enabled	gauge	boolean	Whether the most recently initialized schema selected prefetch
prefetch.parallelism	gauge	readers	Effective reader parallelism
prefetch.read-ranges.total	counter	ranges	Generated ranges before bitmap filtering
prefetch.read-ranges.after-bitmap	counter	ranges	Ranges retained after bitmap filtering
prefetch.seek.count	counter	operations	Underlying reader seek operations
prefetch.produced-batches	counter	batches	Data batches placed into prefetch queues
prefetch.consumed-batches	counter	batches	Data batches returned to the consumer
prefetch.discarded-batches	counter	batches	Data batches released without consumption, plus EOF entries released during cleanup
prefetch.errors	counter	errors	Errors recorded by the background prefetch loop
prefetch.adaptive-disabled-count	counter	decisions	Times adaptive strategy disabled prefetch
prefetch.queue-full-count	counter	events	Times production found a full queue
prefetch.queue-depth	gauge	queue entries	Current queued entries, including retained EOF markers
prefetch.queue-depth.max	gauge	queue entries	Maximum queued entries in the reader lifetime
prefetch.reader-read-latency-us	histogram	microseconds	Underlying reader batch latency
prefetch.consumer-wait-latency-us	histogram	microseconds	Consumer wait latency per returned batch or EOF

Prefetch I/O

PrefetchIoMetrics describes only I/O that passes through the prefetch reader’s instrumented input streams. It is not a whole-query or whole-table I/O total. All counters accumulate for the reader lifetime and are retained across SetReadSchema() and cache reset. Latency uses relaxed atomic count and sum counters instead of per-I/O histograms to reduce hot-path cost. Collection is disabled by default; set prefetch.io-metrics.enabled to true in the read options to enable it. When disabled, these per-I/O metrics are absent and the input streams have no metrics instrumentation. io.async.pending is current state and returns to zero when all callbacks complete. These metrics are C++-only and have no counterparts in Java Paimon.

Name	Type	Unit	Meaning
io.read.requests	counter	requests	Synchronous read requests
io.read.requested-bytes	counter	bytes	Bytes requested by synchronous reads
io.read.physical-bytes	counter	bytes	Bytes returned by successful synchronous reads
io.read.failed	counter	requests	Failed synchronous reads
io.read.latency.count	counter	requests	Completed synchronous read latency samples
io.read.latency.sum-us	counter	microseconds	Sum of synchronous read latency
io.async.requests	counter	requests	Asynchronous read requests
io.async.requested-bytes	counter	bytes	Bytes requested by asynchronous reads
io.async.physical-bytes	counter	bytes	Bytes attributed to successful asynchronous reads
io.async.completed	counter	requests	Successful asynchronous reads
io.async.failed	counter	requests	Failed asynchronous reads
io.async.pending	gauge	requests	Asynchronous callbacks not yet completed
io.async.latency.count	counter	requests	Completed asynchronous callback latency samples
io.async.latency.sum-us	counter	microseconds	Sum of asynchronous callback latency