class
Llama::SamplerChain
- Llama::SamplerChain
- Reference
- Object
Overview
SamplerChain: manages a chain of samplers, but is not itself a Sampler::Base. Ownership and lifecycle are managed internally.
Defined in:
llama/sampler_chain.crConstructors
-
.new(no_perf : Bool = false)
Creates a new SamplerChain with optional parameters
Class Method Summary
-
.open(no_perf : Bool = false, & : self -> _)
Creates a new SamplerChain, yields it to the block, and frees it when the block returns.
Instance Method Summary
-
#accept(token : Int32)
Accepts a token, updating the internal state of the samplers
-
#add(sampler : Sampler::Base)
Adds a sampler to the chain
- #finalize
-
#free : Nil
Releases the underlying C resources, including owned samplers
-
#print_perf
Print performance information for this sampler chain
-
#remove(index : Int)
Removes a sampler from the chain at the given index.
-
#reset
Resets the internal state of the sampler chain.
-
#reset_perf
Reset performance counters for this sampler chain
-
#sample(ctx : Context, idx : Int32 = -1) : Int32
Samples a token using the sampler chain
-
#to_unsafe : Pointer(Llama::LibLlama::LlamaSampler)
For C API compatibility
Constructor Detail
Creates a new SamplerChain with optional parameters
Parameters:
- no_perf: Whether to disable performance counters (default: false)
Raises:
- Llama::Error if the sampler chain cannot be created
Class Method Detail
Creates a new SamplerChain, yields it to the block, and frees it when the block returns.
The resources are released deterministically without calling
#free manually.
Parameters:
- no_perf: Whether to disable performance counters (default: false)
Returns:
- The value of the block
Raises:
- Llama::Error if the sampler chain cannot be created
Instance Method Detail
Accepts a token, updating the internal state of the samplers
Parameters:
- token: The token to accept
Adds a sampler to the chain
Parameters:
- sampler: The sampler to add to the chain
Releases the underlying C resources, including owned samplers
Calling this method multiple times is safe; only the first call releases the resources. The finalizer also calls this method, so manual calls are only needed to release resources deterministically, for example before process exit.
Removes a sampler from the chain at the given index. Returns the original Sampler::Base instance with ownership restored.
Resets the internal state of the sampler chain. Useful when reusing the same chain for multiple independent generations.
Samples a token using the sampler chain
Parameters:
- ctx: The context to sample from
- idx: The index of the logits to sample from (-1 for the last token)
Returns:
- The sampled token