ChatRequestDTO Class Reference

Data Transfer Object for sending chat requests to the LLM. More...

Public Attributes
string	model_id
	The identifier of the model to use.
List< Message >	messages
	The list of in the conversation.
int	n_ctx
	The context window size for model.
bool	f16_kv
	Whether to use 16-bit key/value memory.
int	n_parts
	The number of parts to split the model into.
int	seed
	The random seed for generation.
int	n_gpu_layers
	The number of GPU layers to use. If set to -1, the model will use all available GPU layers. If set to 0, the model will run on CPU. If set to a positive integer, it will use that many GPU layers.
int	max_tokens
	The maximum number of tokens to generate.
float	temperature
	The temperature for sampling.
float	top_p
	The nucleus sampling probability.

Detailed Description

Data Transfer Object for sending chat requests to the LLM.

Member Data Documentation

f16_kv

bool ChatRequestDTO.f16_kv

Whether to use 16-bit key/value memory.

max_tokens

int ChatRequestDTO.max_tokens

The maximum number of tokens to generate.

messages

List<Message> ChatRequestDTO.messages

The list of in the conversation.

model_id

string ChatRequestDTO.model_id

The identifier of the model to use.

n_ctx

int ChatRequestDTO.n_ctx

The context window size for model.

n_gpu_layers

int ChatRequestDTO.n_gpu_layers

The number of GPU layers to use. If set to -1, the model will use all available GPU layers. If set to 0, the model will run on CPU. If set to a positive integer, it will use that many GPU layers.

n_parts

int ChatRequestDTO.n_parts

The number of parts to split the model into.

seed

int ChatRequestDTO.seed

The random seed for generation.

temperature

float ChatRequestDTO.temperature

The temperature for sampling.

top_p

float ChatRequestDTO.top_p

The nucleus sampling probability.

---

The documentation for this class was generated from the following file:

• RaymondMaarloeve/Assets/Scripts/LLM/DTO/ChatDTO.cs