loader

triplum.utils.data.loader

Lazy batching and customizable collation, independent of domain schemas.

DataLoader

DataLoader(
    dataset: Iterable[Item],
    *,
    batch_size: int = 1,
    collate_fn: None = None,
)

DataLoader(
    dataset: Iterable[Item],
    *,
    batch_size: None,
    collate_fn: None = None,
)

DataLoader(
    dataset: Iterable[T],
    *,
    batch_size: int = 1,
    collate_fn: Callable[[list[T]], B],
)

DataLoader(
    dataset: Iterable[T],
    *,
    batch_size: int | None = 1,
    collate_fn: Callable[[list[T]], B] | None = None,
)

Consume a source lazily, yielding lists or batches made by collate_fn.

batch_size=None passes through source items unchanged, including native batches. The loader neither prefetches nor replays a one-shot iterator. It keeps the final partial batch and lets source and collator exceptions propagate.

Source code in src/triplum/utils/data/loader.py

def __init__(
    self,
    dataset: Iterable[T],
    *,
    batch_size: int | None = 1,
    collate_fn: Callable[[list[T]], B] | None = None,
) -> None:
    if batch_size is not None and (isinstance(batch_size, bool) or batch_size < 1):
        raise ValueError("batch_size must be a positive integer or None")
    if batch_size is None and collate_fn is not None:
        raise ValueError("collate_fn requires automatic batching")
    self.dataset = dataset
    self.batch_size = batch_size
    self.collate_fn = collate_fn

dataset instance-attribute

dataset = dataset

batch_size instance-attribute

batch_size = batch_size

collate_fn instance-attribute

collate_fn = collate_fn