class Tiktoken::Encoding

• Tiktoken::Encoding
• Reference
• Object

Defined in:

tiktoken/encoding.cr

Class Method Summary

• .cl100k_base
• .for_model(model_name : String)
• .o200k_base
• .o200k_harmony
• .p50k_base
• .p50k_edit
• .r50k_base

Instance Method Summary

• #decode(tokens) : String
• #encode(text : String, allowed_special : Set(String) = Set(String).new)
• #encode_ordinary(text : String)
• #encode_with_special_tokens(text : String)
• #finalize

Class Method Detail

Instance Method Detail