ExLLama.Model (LLama CPP Nif Wrapper v0.4.1)

Copy Markdown

A loaded GGUF model handle wrapping a llama.cpp model resource (C NIF, bundled llama.cpp v0.6.0).

Load with ExLLama.load_model/2 or load_from_file/2 — loading options (load_mode, lazy_mode, split_mode, n_gpu_layers, ...) are an ExLLama.ModelOptions struct; fields left nil preserve upstream defaults.

What a model can do

The struct is an immutable handle and may be shared across processes; each session owns its own context/KV cache.

Summary

Functions

bos(model)

create_session(model)

create_session(model, options)

decode_tokens(model, tokens)

detokenize(model, token)

embed_len(model)

embeddings(model, inputs, options)

eos(model)

eot(model)

infill_middle(model)

infill_prefix(model)

infill_suffix(model)

load_from_file(path)

load_from_file(path, opts)

nl(model)

token_to_byte_piece(model, token)

token_to_piece(model, token)

train_len(model)

vocabulary_size(model)