class Llama::Sampler::Grammar

Overview

Grammar sampler

The Grammar sampler constrains token generation to follow a formal grammar defined in GBNF format. This is useful for generating structured text like JSON, XML, or code that must follow specific syntax rules.

Example:

# Define a simple grammar for a numbered list
grammar = %q{
  root ::= list
  list ::= item+
  item ::= number " " text "\n"
  number ::= "1" | "2" | "3" | "4" | "5"
  text ::= [a-zA-Z ]+
}

sampler = Llama::Sampler::Grammar.new(model.vocab, grammar, "root")

Defined in:

llama/sampler/grammar.cr

Constructors

Instance Method Summary

Instance methods inherited from class Llama::Sampler::Base

to_unsafe : Pointer(Llama::LibLlama::LlamaSampler) to_unsafe

Constructor methods inherited from class Llama::Sampler::Base

new(handle : Pointer(LibLlama::LlamaSampler)) new

Constructor Detail

def self.new(vocab : Vocab, grammar_str : String, grammar_root : String) #

Creates a new Grammar sampler

Parameters:

  • vocab: The vocabulary to use
  • grammar_str: The grammar definition string in GBNF format
  • grammar_root: The root symbol of the grammar

Raises:

  • Llama::Error if the sampler cannot be created

[View source]

Instance Method Detail

def finalize #

Overrides the parent class's finalize method to ensure proper cleanup


[View source]