class Tiktoken::Encoding

• Tiktoken::Encoding
• Reference
• Object

Defined in:

tiktoken/encoding.cr

Class Method Summary

• .for_model(model_name : String)

Instance Method Summary

• #decode(tokens) : String
• #encode(text : String, allowed_special : Set(String) = Set(String).new)
• #encode_ordinary(text : String)
• #encode_with_special_tokens(text : String)
• #finalize

Class Method Detail

Instance Method Detail