chunking

triplum.steps.chunking

Chunker

Bases: Protocol

Split one source into chunks whose start and text are exact slices of its text.

FixedSize

FixedSize(size: int)

Bases: Chunker, FingerprintedComputationMixin

Slice the text every size characters; the last chunk keeps the remainder.

Chunks do not overlap and concatenate back to the source text. Empty text yields no chunks.

Source code in src/triplum/steps/chunking.py

def __init__(self, size: int) -> None:
    if isinstance(size, bool) or size < 1:
        raise ValueError("size must be a positive integer")
    self.size = size

size instance-attribute

size = size

fingerprint_config

fingerprint_config() -> dict[str, object]

Source code in src/triplum/steps/chunking.py

def fingerprint_config(self) -> dict[str, object]:
    return {"size": self.size}