dag-ml-data¶
dag-ml-data is the Rust-first data-contract and planning layer for typed,
sample-aligned, multi-source ML data. It owns schemas, semantic axes,
representations, immutable data views, sample relations, representation
adapters, data plans, alignment/collation contracts, schema fingerprints and
the host-provider C ABI. ML phases, OOF joins and model execution belong to the
companion dag-ml repo.
This site is the contributor and integration entry point for data contracts used
by future nirs4all and nirs4all-core integrations. The nirs4all connector is
owned by nirs4all-io; this repo stays NIRS-agnostic.
Start Here¶
Need |
Page |
|---|---|
Build and validate locally |
|
Understand data/runtime boundaries |
|
Integrate a data provider over C ABI |
|
Check shipped vs pending scope |
|
Run the documented gates |
|
Review roadmap and release gates |
|
Check the supported release surface |
|
Map aggregation with |
|
Run release performance probes |
|
Map nirs4all replacement data capabilities |
|
Inspect shared contracts |
|
Review shared decisions |
|
Pick an example by audience |
|
API References¶
Rust core API: https://docs.rs/dag-ml-data-core/latest/
Rust facade API: https://docs.rs/dag-ml-data/latest/
C ABI source:
crates/dag-ml-data-capi/include/dag_ml_data.hPython binding source:
crates/dag-ml-data-pyWASM binding source:
crates/dag-ml-data-wasm
The nirs4all ecosystem¶
The ML coordinator that consumes this data layer (shared, JSON-identical contracts).
Main Python modelling library — pipelines, SpectroDataset, predictions.
Portable C-ABI PLS/NIRS engine (libn4m) + bindings.
Rust readers for ~58 NIRS/spectroscopy file formats.
Dataset-assembly bridge → SpectroDataset.
Curated DOI-pinned NIRS dataset catalog (n4a-datasets).
Canonical portable aggregate distribution (Rust, Python, R, WASM, MATLAB/Octave).
