Index A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | R | S | T | V | W | Z A Accumulator (class in zephon.ops) accumulator() (BaseOp method) add_op() (Pipeline method) aggregate_compressor (RuntimeOptions attribute) aggregate_dir (RuntimeOptions attribute) aggregate_serializer (RuntimeOptions attribute) aggregate_timeout_s (RuntimeOptions attribute) allow_latency_flush_in_deterministic (RuntimeOptions attribute) apply() (FetchStageSummary method) (FetchTimingSummary method) (FetchTimingTotals method) (PipelineSummary method) (PrefetchTimingSummary method) (PrefetchTimingTotals method) as_cursor_key() (SampleMeta method) AS_IS (ComponentOrder attribute) as_key() (SampleCursor method) auto_validation (RuntimeOptions attribute) avg_close_ns (FetchTimingTotals property) avg_group_ns (FetchTimingTotals property) avg_open_ns (FetchTimingTotals property) avg_read_ns (FetchTimingTotals property) avg_resolve_ns (FetchTimingTotals property) B backend (Dataset attribute) base_offset (SampleCursor property) BaseOp (class in zephon.ops) batch() (Pipeline method) batch_shape_sensitive (OpTraits attribute) batch_size (PrefetchTimingDelta attribute) batches (PrefetchTimingTotals attribute) BOTH (MetricsSinkMode attribute) build_index() (in module zephon.build_index) C cache (StoreOptions attribute) cache_hit_ratio (FetchTimingTotals property) cache_hits (FetchTimingDelta attribute) (FetchTimingTotals attribute) cache_misses (FetchTimingDelta attribute) (FetchTimingTotals attribute) CacheOptions (class in zephon.io) calibration_samples (TokenEstimation attribute) calibration_shards_max (TokenEstimation attribute) calibration_shards_min (TokenEstimation attribute) canonical_replicas (RuntimeOptions attribute) capacity (MTPQueueStats attribute) checkpoint() (Pipeline method) child() (SampleCursor method) (SampleMeta method) chunk_id (SampleCursor attribute) (SampleMeta attribute) chunk_ids (SampleBatch attribute) chunk_offset (SampleCursor attribute) (SampleMeta attribute) chunk_size_hint() (StaticMixtureWorkSource method) (WorkSource method) ChunkId (in module zephon.types) ChunkOffset (in module zephon.types) clone() (FetchTimingSummary method) (PipelineSummary method) (PrefetchTimingSummary method) clone_for_lane() (StaticMixtureWorkSource method) (WorkSource method) close() (InMemoryShard method) close_ns (FetchTimingDelta attribute) (FetchTimingTotals attribute) coalesce_tensors (RuntimeOptions attribute) code (Issue attribute) collect_stats (StageInfo attribute) collects_nodes (ExecutionTrackingMode property) collects_stages (ExecutionTrackingMode property) component_ids() (StaticMixtureWorkSource method) (WorkSource method) component_sample_counts (SampleMeta attribute) component_token_counts (SampleMeta attribute) ComponentId (in module zephon.types) ComponentOrder (class in zephon.work) components (WorkChunk attribute) compute_wait_ratios() (PipelineSummary method) contribution_cursors() (SampleMeta method) contribution_refs() (SampleMeta method) ContributorRef (class in zephon.types) contributors (SampleMeta property) copy() (FetchStageSummary method) (FetchTimingTotals method) (PrefetchTimingTotals method) CountingAccumulator (class in zephon.ops) counts() (Dataset method) cursor (ContributorRef attribute) (SampleMeta property) D Dataset (class in zephon.io) dataset_id (FetchTimingDelta attribute) dataset_ids (StaticMixtureWorkSource property) DatasetId (in module zephon.types) datasets_by_id (StaticMixtureWorkSource property) (WorkSource property) decode_text() (Pipeline method) default_stage_prefetch (RuntimeOptions attribute) default_tags (MetricsSinkConfig attribute) depth (MTPQueueStats attribute) deterministic (RuntimeOptions attribute) doc_link (Issue attribute) DomainGroups (class in zephon.ops) download_retry (CacheOptions attribute) download_timeout (CacheOptions attribute) dp_degree (RuntimeOptions attribute) dp_group_id (RuntimeOptions attribute) E enable_observability() (Pipeline method) enabled (CacheOptions attribute) (ParquetRGCacheOptions attribute) ensure_mixture() (Pipeline method) errors (ValidationReport property) execution_tracking (RuntimeOptions attribute) ExecutionTrackingMode (class in zephon.observability) explain() (Pipeline method) F fallback_tokens_per_byte (TokenEstimation attribute) fetch() (Pipeline method) fetch_parallelism() (Pipeline method) fetch_timing_snapshot() (Pipeline method) FetchStageSummary (class in zephon.observability) FetchTimingDelta (class in zephon.observability) FetchTimingSummary (class in zephon.observability) FetchTimingTotals (class in zephon.observability) flush() (Accumulator method) (CountingAccumulator method) (PassthroughAccumulator method) flush_every_k_chunks (RuntimeOptions attribute) flush_interval_s (MetricsSinkConfig attribute) format() (ValidationReport method) from_any() (CacheOptions class method) (ParquetRGCacheOptions class method) (StoreOptions class method) from_dict() (Dataset class method) from_key() (SampleCursor static method) from_path() (Dataset class method) from_state() (WorkChunk class method) from_value() (ExecutionTrackingMode class method) (MetricsSinkMode class method) G get() (OpContext method) getsamples() (InMemoryShard method) global_rank (RuntimeOptions attribute) group_ns (FetchTimingDelta attribute) (FetchTimingTotals attribute) groups (FetchTimingTotals attribute) H has_pending_data() (Accumulator method) (CountingAccumulator method) has_samples() (FetchTimingSummary method) (PrefetchTimingSummary method) I ids (SampleBatch property) ids() (Dataset method) includes_logs() (MetricsSinkMode method) includes_statsd() (MetricsSinkMode method) index (FetchStageSummary attribute) indexable (OpTraits attribute) inflight_summary() (Pipeline method) InMemoryShard (class in zephon.io) io_options (RuntimeOptions attribute) ipc_buffer_bytes (RuntimeOptions attribute) ipc_transport (RuntimeOptions attribute) is_flush_sentinel (SampleMeta property) is_indexable (Pipeline property) is_last_child (ContributorRef attribute) is_sentinel (SampleMeta property) Issue (class in zephon.validation) issues (ValidationReport attribute) iter_samples() (WorkChunk method) iter_stages() (FetchTimingSummary method) (PipelineSummary method) J json_logs (MetricsSinkConfig attribute) K keep_zip (CacheOptions attribute) L lane_id (SampleMeta attribute) lane_ids (SampleBatch attribute) LaneId (in module zephon.types) limit_bytes (CacheOptions attribute) (ParquetRGCacheOptions attribute) lineage (SampleCursor attribute) (SampleMeta attribute) lineage_paths (SampleBatch property) LineageIndex (in module zephon.types) load_state_dict() (StaticMixtureWorkSource method) (WorkSource method) LocalSampleId (in module zephon.types) LOG (MetricsSinkMode attribute) M map_batch() (Pipeline method) map_transform() (Pipeline method) mapping_strategy (RuntimeOptions attribute) materialize() (Pipeline method) materialize_order() (WorkChunk method) max_batch_size (MetricsSinkConfig attribute) max_count() (Dataset method) max_slack_bytes (CacheOptions attribute) max_worker_retries (RuntimeOptions attribute) max_workers (RuntimeOptions attribute) measure (TokenEstimation attribute) merge() (CacheOptions method) (FetchStageSummary method) (FetchTimingSummary method) (FetchTimingTotals method) (ParquetRGCacheOptions method) (PipelineSummary method) (PrefetchTimingSummary method) (PrefetchTimingTotals method) (StoreOptions method) message (Issue attribute) meta (SampleRecord attribute) metrics_sink_config (RuntimeOptions attribute) metrics_snapshot() (Pipeline method) MetricsSinkConfig (class in zephon.observability) MetricsSinkMode (class in zephon.observability) min_free_bytes (ParquetRGCacheOptions attribute) min_slack_bytes (CacheOptions attribute) mixture (WorkChunk property) mixture_config (RuntimeOptions attribute) MixtureReadConfig (class in zephon.work) MixtureReadMode (class in zephon.work) MixtureSpec (class in zephon.work) mode (MetricsSinkConfig attribute) (MixtureReadConfig attribute) module zephon.build_index zephon.io zephon.observability zephon.ops zephon.options zephon.types zephon.validation zephon.work mp_context (RuntimeOptions attribute) mtp_auto_checkpoint (RuntimeOptions attribute) mtp_buffer (RuntimeOptions attribute) mtp_buffer_bytes (RuntimeOptions attribute) mtp_mode (RuntimeOptions attribute) mtp_prefetch (RuntimeOptions attribute) mtp_queue_stats() (Pipeline method) MTPQueueStats (class in zephon.observability) N name (Dataset attribute) namespace (MetricsSinkConfig attribute) next_chunk() (StaticMixtureWorkSource method) (WorkSource method) NODES (ExecutionTrackingMode attribute) normalized (MixtureSpec property) normalized_for() (MixtureSpec method) O OFF (ExecutionTrackingMode attribute) ok (ValidationReport property) op_index (StageInfo attribute) op_name (Issue attribute) op_queue_capacity (RuntimeOptions attribute) OpContext (class in zephon.ops) open_ns (FetchTimingDelta attribute) (FetchTimingTotals attribute) open_retry_attempts (CacheOptions attribute) open_retry_initial_backoff (CacheOptions attribute) open_retry_max_backoff (CacheOptions attribute) options() (Pipeline method) OpTraits (class in zephon.ops) P pack_flat() (Pipeline method) pack_meta() (in module zephon.ops) pack_sequences() (Pipeline method) padding_length (SampleMeta property) parallelism (OpTraits attribute) parquet_rg_cache (StoreOptions attribute) ParquetRGCacheOptions (class in zephon.io) PassthroughAccumulator (class in zephon.ops) path (Dataset attribute) payload (SampleRecord attribute) per_stage_prefetch (RuntimeOptions attribute) per_stage_runner (RuntimeOptions attribute) Pipeline (class in zephon) PipelineSummary (class in zephon.observability) plan_id (FetchTimingSummary attribute) (PipelineSummary attribute) (PrefetchTimingSummary attribute) plan_identity() (BaseOp class method) precompute (MixtureReadConfig attribute) prefetch() (Pipeline method) prefetch_batches (RuntimeOptions attribute) prefetch_depth (MTPQueueStats attribute) prefetch_failed (PrefetchTimingDelta attribute) (PrefetchTimingTotals attribute) prefetch_requests (PrefetchTimingDelta attribute) (PrefetchTimingTotals attribute) prefetch_succeeded (PrefetchTimingDelta attribute) (PrefetchTimingTotals attribute) prefetch_timing_snapshot() (Pipeline method) PrefetchTimingDelta (class in zephon.observability) PrefetchTimingSummary (class in zephon.observability) PrefetchTimingTotals (class in zephon.observability) preflight_tokenizers() (in module zephon.validation) (Pipeline method) preserves_cursor_order (OpTraits attribute) prime() (StaticMixtureWorkSource method) (WorkSource method) primer (TokenEstimation attribute) process_many() (BaseOp method) process_one() (BaseOp method) push_many() (Accumulator method) (CountingAccumulator method) (PassthroughAccumulator method) R raw_bytes (InMemoryShard property) raw_bytes() (Dataset method) read_ns (FetchTimingDelta attribute) (FetchTimingTotals attribute) reads_payload (Accumulator property) records (SampleBatch attribute) reporting_interval_s (FetchTimingSummary attribute) (PipelineSummary attribute) (PrefetchTimingSummary attribute) requires_serial_state (OpTraits attribute) requires_token_priming (StaticMixtureWorkSource property) (WorkSource property) resolve_ns (FetchTimingDelta attribute) (FetchTimingTotals attribute) resolved_parquet_rg_cache() (StoreOptions method) restore() (Pipeline method) retries (FetchTimingDelta attribute) (FetchTimingTotals attribute) rg_cache_bytes (CacheOptions attribute) root (CacheOptions attribute) (ParquetRGCacheOptions attribute) run_id (RuntimeOptions attribute) runner (RuntimeOptions attribute) RuntimeOptions (class in zephon.options) S sample_at() (WorkChunk method) sample_id (SampleCursor attribute) (SampleMeta attribute) sample_id_at() (StaticMixtureWorkSource method) (WorkSource method) SampleBatch (class in zephon.types) SampleCursor (class in zephon.types) SampleMeta (class in zephon.types) SamplePayloadArray (in module zephon.types) SampleRecord (class in zephon.types) samples (FetchTimingDelta attribute) (FetchTimingTotals attribute) (PrefetchTimingTotals attribute) seed (MixtureReadConfig attribute) (WorkChunk attribute) setup() (BaseOp method) severity (Issue attribute) shard_count() (Dataset method) shard_id (FetchTimingDelta attribute) shard_reopens (FetchTimingDelta attribute) (FetchTimingTotals attribute) shard_totals (FetchStageSummary attribute) ShardId (in module zephon.types) shm_min_size (RuntimeOptions attribute) should_emit_statsd() (MetricsSinkConfig method) should_log() (MetricsSinkConfig method) SHUFFLE (ComponentOrder attribute) shuffle() (Pipeline method) shutdown_mode (RuntimeOptions attribute) spawn_child() (in module zephon.ops) stage_index (FetchTimingDelta attribute) (PrefetchTimingDelta attribute) (StageInfo attribute) stage_name (StageInfo attribute) staged_bytes (MTPQueueStats attribute) StageInfo (class in zephon.ops) STAGES (ExecutionTrackingMode attribute) stages (FetchTimingSummary attribute) (PipelineSummary attribute) (PrefetchTimingSummary attribute) state_dict() (StaticMixtureWorkSource method) (WorkChunk method) (WorkSource method) stateful_transform() (Pipeline method) StaticMixtureWorkSource (class in zephon.work) STATSD (MetricsSinkMode attribute) statsd_host (MetricsSinkConfig attribute) statsd_port (MetricsSinkConfig attribute) StoreOptions (class in zephon.io) success_rate (PrefetchTimingSummary property) supports_indexing() (StaticMixtureWorkSource method) (WorkSource method) T tags (SampleMeta attribute) target_mixture (WorkChunk attribute) to_indexable_torch_dataset() (Pipeline method) to_member_map() (DomainGroups method) to_records() (FetchStageSummary method) (FetchTimingSummary method) (PipelineSummary method) to_torch_dataset() (Pipeline method) to_training() (SampleBatch method) TokenEstimation (class in zephon.work) tokenize() (Pipeline method) tokenize_chat() (Pipeline method) tombstone (SampleMeta property) tombstone_meta() (in module zephon.ops) tombstones_for_record() (in module zephon.ops) total() (Dataset method) total_samples (StaticMixtureWorkSource property) total_wait_ratio (PipelineSummary attribute) totals (FetchStageSummary attribute) tracking_mode (FetchTimingSummary attribute) (PipelineSummary attribute) (PrefetchTimingSummary attribute) traits() (BaseOp method) try_epoch_reset() (Accumulator method) V validate() (Pipeline method) validate_against() (DomainGroups method) validate_for() (MixtureSpec method) validate_hash (CacheOptions attribute) validation_samples() (BaseOp method) ValidationError ValidationReport (class in zephon.validation) W warnings (ValidationReport property) WEIGHTED_RANDOM (MixtureReadMode attribute) WEIGHTED_ROUND_ROBIN (MixtureReadMode attribute) weights (MixtureSpec attribute) with_additional_tags() (MetricsSinkConfig method) with_contributors() (SampleMeta method) with_lineage() (SampleMeta method) with_tombstone() (SampleMeta method) within_component (MixtureReadConfig attribute) WorkChunk (class in zephon.work) worker_allocation (RuntimeOptions attribute) WorkSource (class in zephon.work) world_size (RuntimeOptions attribute) Z zephon.build_index module zephon.io module zephon.observability module zephon.ops module zephon.options module zephon.types module zephon.validation module zephon.work module