Read recipe results for analysis¶

After a recipe run succeeds, you usually want tables for analysis or training — not a file browser.

Nexus offers two read paths. Choose from the recipe you just ran:

After this recipe outcome…	Start here
Dataset-generator wrote a published table	Published tables
Converter (or upload) left Parquet or other files as collection items	Files in a collection

Adapters: pandas (Dataframe), PyTorch (Torch).

Read files in a collection¶

Use this when tabular files live as items in a target collection — for example Parquet written or uploaded into the collection.

from miura.adapters import DatasetSource
from miura.adapters.dataframe import to_dataframe

source = DatasetSource.from_collection(target_collection, path="/")

df = to_dataframe(
    source,
    columns=["temperature", "pressure", "label"],
    limit_rows=10_000,
)
print(df.head())

   temperature  pressure  label
0         20.0    1010.0      1
1         21.0    1012.0      0
2         22.0    1014.0      1

to_dataframe also accepts a raw collection. Runnable: parquet.py.

ItemSelect filters collection items by the same item attributes you set in Attach metadata (split, labels). That is read-time filtering of files — distinct from CaseSelection, which selects admitted cases when a recipe runs (see Selection).

Read published tables¶

Use this when a dataset-generator recipe (or Parquet upload) published tabular output on a collection. Resolve the asset from the target collection, then read it:

import asyncio

from miura.adapters import DatasetSource
from miura.adapters.dataframe import to_dataframe_async

asset = target_collection.catalog_asset()
source = DatasetSource.from_catalog_asset(
    asset,
    columns=["temperature", "pressure"],
    catalog_batch_size=4096,
)
df = asyncio.run(to_dataframe_async(source, limit_rows=10_000))

A collection of files is not automatically a published table — publication is a recipe or upload outcome. For an explicit CatalogAssetRef, use asset.logical_ref. Runnable: catalog_stream.py (live Flight stream). Wiring-only: catalog.py. API: Published tables.

Build a PyTorch DataLoader¶

Collection and catalog torch adapters are not at feature parity yet.

	`miura.adapters.torch`	`miura.adapters.catalog.torch`
Source	Parquet items in a collection	Catalog asset (published table)
Split / label filter	`select=ItemSelect(...)`	Not supported yet
Path scope	`path="/subdir"`	N/A (column projection only)
Shuffle	Client-side buffer (`shuffle=True`)	Server-side CRS (`shuffle=True`, `epoch=N` per epoch)

Collection — filter by item attributes, shuffle in the client:

from miura.api.annotations import ItemSelect
from miura.adapters.torch import to_torch_dataloader

loader = to_torch_dataloader(
    collection,
    select=ItemSelect(split="train"),
    feature_columns=["temperature", "pressure"],
    target_column="label",
    batch_size=32,
    shuffle=True,
)

Published table — stream published rows; reshuffle each epoch on the server:

from miura.adapters.catalog.torch import to_torch_dataloader

loader = to_torch_dataloader(
    asset_ref,
    feature_columns=["temperature", "pressure"],
    target_column="label",
    batch_size=32,
    shuffle=True,
    epoch=0,
)

Until catalog adapters support split selection, keep train/val/test in separate assets or filter when you publish. See adapters.py and catalog_stream.py.

Next step¶

You can read recipe outputs as collection files or published tables, and plug them into pandas or PyTorch.

For the full API surface, see Published tables and the API reference. Runnable scripts: catalog_stream.py, Example scripts.