class Llama::SamplerChain

Overview

SamplerChain: manages a chain of samplers, but is not itself a Sampler::Base. Ownership and lifecycle are managed internally.

Included Modules

Defined in:

llama/sampler_chain.cr

Constructors

Class Method Summary

Instance Method Summary

Constructor Detail

def self.new(no_perf : Bool = false) #

Creates a new SamplerChain with optional parameters

Parameters:

  • no_perf: Whether to disable performance counters (default: false)

Raises:

  • Llama::Error if the sampler chain cannot be created

[View source]

Class Method Detail

def self.open(no_perf : Bool = false, & : self -> _) #

Creates a new SamplerChain, yields it to the block, and frees it when the block returns.

The resources are released deterministically without calling #free manually.

Parameters:

  • no_perf: Whether to disable performance counters (default: false)

Returns:

  • The value of the block

Raises:

  • Llama::Error if the sampler chain cannot be created

[View source]

Instance Method Detail

def accept(token : Int32) #

Accepts a token, updating the internal state of the samplers.

This is intended for manual sampling paths that use sampler apply; a token returned by #sample has already been accepted.

Parameters:

  • token: The token to accept

[View source]
def add(sampler : Sampler::Base) #

Adds a sampler to the chain

Parameters:

  • sampler: The sampler to add to the chain

[View source]
def close : Nil #

Releases the chain and every sampler whose ownership it accepted.


[View source]
def closed? : Bool #

Returns whether the sampler chain has been released.


[View source]
def finalize #

[View source]
def free : Nil #

Releases the underlying C resources, including owned samplers

Calling this method multiple times is safe; only the first call releases the resources. The finalizer also calls this method, so manual calls are only needed to release resources deterministically, for example before process exit.


[View source]
def print_perf #

Print performance information for this sampler chain


[View source]
def remove(index : Int) #

Removes a sampler from the chain at the given index. Returns the original Sampler::Base instance with ownership restored.


[View source]
def reset #

Resets the internal state of the sampler chain. Useful when reusing the same chain for multiple independent generations.


[View source]
def reset_perf #

Reset performance counters for this sampler chain


[View source]
def sample(ctx : Context, idx : Int32 = -1) : Int32 #

Samples and accepts a token using the sampler chain

Parameters:

  • ctx: The context to sample from
  • idx: The index of the logits to sample from (-1 for the last token)

Returns:

  • The sampled token

llama.cpp's llama_sampler_sample applies the chain, selects a token, and accepts it. Do not call #accept again for the returned token.


[View source]
def to_unsafe : Pointer(Llama::LibLlama::LlamaSampler) #

For C API compatibility


[View source]
def unsafe_handle! : Pointer(LibLlama::LlamaSampler) #

Returns a checked native handle for internal wrapper use. :nodoc:


[View source]