# Roadmap ## 0.2.x RC Release Closure The 0.2.x RC release surface is closed around data contracts, provider conformance, C ABI/Python/WASM JSON-contract bindings, cross-repo `dag-ml` fixtures and release gates documented in `docs/SUPPORTED.md`. Remaining items in this roadmap are post-0.2.x hardening/backlog unless a later section explicitly says otherwise. ## Phase 0: Contracts Frozen Definition of done: - Rust types for ids, axes, representations, sources, schemas, views and plans; - deterministic schema fingerprint; - C ABI helper for fingerprinting; - source design docs moved into `docs/design/source`; - first CLI and tests pass. ## Phase 1: Path Solver Definition of done: - adapter registry; - BFS/Dijkstra representation path search; - refusal and `requires_user_choice` reporting; - fixture tests for tabular, dense signal, image and time-series paths. Status: implemented for the current fixtures, including deterministic model-input planning, coordinator envelope export, explicit envelope schema versioning and a published v1 JSON Schema artifact shared and compared with `dag-ml` in CI. A shared conformance-pack manifest now pins the normalized schema digests, canonical fixture digests, C ABI requirements and required cross-repo checks. ## Phase 2: Host Providers Definition of done: - in-memory Rust provider for tests; - Python provider skeleton and reusable `ctypes` smoke; - C ABI conformance around view creation and identity/target/feature export; - Arrow identity, target and feature table smoke tests. Status: first in-memory coordinator handle arena implemented. It validates a coordinator envelope plus materialization request, returns an opaque handle record, and records run/node/phase/variant/fold/fingerprint traceability for `dag-ml` controller tasks. It also creates identity-filtered view handles and aligns sample-level target values while de-duplicating repeated observations. The C ABI now exposes a Rust-owned in-memory provider vtable that materializes data handles, creates view handles, exports view identity, exports numeric targets, exports numeric observation-level features, and supports release/destroy callbacks. A stdlib-only Python example now exercises the same lifecycle through the public ABI. Native vtable conformance now checks materialize/view/export paths plus release behavior, including parent-handle release invalidating child views. Provider feature inputs are converted once at provider construction into reusable column-major `NumericFeatureBuffer` values from `dag-ml-data-core`, grouped in a `NumericFeatureBufferArena`, and exposed through deterministic feature-buffer manifests before Arrow export. The preferred numeric conformance inputs are typed row-major `NumericFeatureMatrixF64` values and borrowed C `DagMlDataFeatureMatrixF64View` descriptors with optional validity masks, avoiding per-cell JSON numeric value transport. The borrowed C path also accepts column-major `DagMlDataFeatureMatrixF64ColumnarView` descriptors with per-column f64 slices and optional per-column validity bitmaps, mirroring Arrow/Parquet/NumPy columnar layouts and avoiding the row-major transpose copy. The arena binds compatible buffers to each materialized data handle after source-scoped relations are known, and provider feature/fusion/collation exports validate those bindings before exporting Arrow or tensors. Next: extend the tensor ABI beyond f64 row-major blocks using the same data-handle binding contract. ## Phase 3: Alignment, Fusion And Collation Definition of done: - `inner`, `left`, `outer` alignment plans; - presence-mask propagation; - feature joiner contracts; - late collation contracts for dense tensor models. Status: started. The core now has a deterministic sample-alignment planner for `inner`, `left` and `outer` policies, and model-input planning emits an explicit `Align` step before multi-source joins. The core also has an executable observation-level feature-fusion kernel for aligned feature blocks. It preserves reference-source repetitions, broadcasts singleton non-reference rows, fills missing outer/left values with nulls, namespaces columns by default and refuses ambiguous repeated non-reference joins. A C ABI conformance helper exports that kernel as Arrow over already materialized feature blocks, and the in-memory provider vtable can route `feature_arrow` JSON fusion selectors through the same kernel using provider-owned typed feature buffers. A core numeric collation kernel now produces row-major tensor blocks with deterministic padding, truncation, presence masks and value-validity masks. The in-memory provider can now route feature-collation JSON selectors, including fused selectors, through provider-owned typed buffers and return deterministic `NumericTensorBlock` JSON or ABI-owned `DagMlDataTensorF64` buffers. The collation ABI now also exposes owned row-major `DagMlDataTensorF32` buffers beside the f64 path. The kernel still operates in f64 to preserve canonical numeric semantics; values are cast to f32 at the ABI boundary and the call is rejected when any value would round-trip to a non-finite f32. Production provider arenas are still pending. ## Phase 4: nirs4all Connector — DESCOPED Status: **descoped, owned by `nirs4all-io`**. See `docs/ADR-0001-nirs4all-connector-ownership.md` for the rationale. `dag-ml-data` retains its NIRS-agnostic invariant; `nirs4all-io` is the sole crate that converts `SpectroDataset` into a `CoordinatorDataPlanEnvelope` and depends on `dag-ml-data` for the envelope contract. The original deliverables (`SpectroDataset` connector, dense signal representations, compatibility tests) move to `nirs4all-io`'s Phase 2 backlog. The only `dag-ml-data` change pulled in by this descoping is the generic spectroscopy axis `AxisKind::Wavenumber`, which unblocks IR and Raman use cases without adding `nirs4all`-specific business logic to core. ## Phase 5: Bundle Replay Definition of done: - serialized `DataPlan` and fitted adapter references; - schema fingerprint compatibility checks; - replay fixtures shared with `dag-ml`.