ExLLama.SessionOptions (LLama CPP Nif Wrapper v0.4.1)

Copy Markdown

Options used when creating an inference session (context).

See ExLLama.ContextParams for the meaning of the numeric enum fields (flash_attn_type, pooling_type, attention_type, type_k, type_v, ...). Unset fields fall back to llama.cpp defaults — only explicitly provided keys are forwarded to the NIF.

Summary

Functions

new()

new(x)