class Llama::Session

• Llama::Session
• Reference
• Object

Overview

Stateful text facade. The canonical transcript is rebuilt before each turn; this conservative strategy keeps stop-truncated text and native KV state in agreement until incremental token reconciliation is introduced.

Included Modules

• Llama::NativeResource

Defined in:

llama/session.cr

Constructors

• .new(model : Model, context_options : ContextOptions = ContextOptions.new)

Instance Method Summary

• #close : Nil
• #closed? : Bool
• #finalize
• #free : Nil
• #generate(prompt : String, options : GenerationOptions = GenerationOptions.new, &block : GenerationChunk -> ) : Generation
• #generate(prompt : String, options : GenerationOptions = GenerationOptions.new) : Generation
• #load(path : String) : Nil
• #reset : Nil
• #restore(value : SessionSnapshot) : Nil

  Restores canonical text only after every compatibility check succeeds.

• #save(path : String) : Nil
• #snapshot : SessionSnapshot
• #to_unsafe
• #transcript_token_count(add_special : Bool = true, parse_special : Bool = true) : Int32

  Returns logical transcript tokens.

• #used_tokens : Int32

  Returns the historical tokenization count, including the vocabulary's default special-token behavior.

Constructor Detail

Instance Method Detail