LlmManager Class Reference

Manages communication with the LLM server, including model registration, loading, unloading, and chat requests. Handles request queuing with parallel execution support. More...

Public Member Functions
void	Setup (string api)
	Sets up the LLM manager with the specified API base URL.
IEnumerator	Get< T > (string endpoint, Action< T > onSuccess, Action< string > onError)
	Sends a GET request to the specified endpoint and deserializes the response to type T.
void	QueuePostRequest< T, TRequest > (string endpoint, TRequest data, Action< T > onSuccess, Action< string > onError)
	Queues a POST request. Execution starts if the number of active requests is below MaxParallelRequests.
void	Status (Action< StatusDTO > onComplete, Action< string > onError)
	Gets the status of the LLM server.
void	Chat (string modelID, List< Message > messages, Action< ChatResponseDTO > onComplete, Action< string > onError, float top_p=0.95f, float temperature=0.8f, int maxTokens=4096)
	Sends a chat request to the LLM server using the specified model and message history.
void	Connect (Action< bool > onComplete)
	Connects to the LLM server and unloads all currently loaded models.
void	GenericComplete (MessageDTO message)
	Generic callback for handling MessageDTO responses, logs success or error messages.

Static Public Member Functions
static bool	StatusCommand ()
	Console command to check the status of the LLM server.
static bool	LLMQueue ()

Public Attributes
int	MaxParallelRequests = 4
	Maximum number of concurrent POST requests allowed.
bool	LogDebug = false
	Whether log requests and responses.

Static Public Attributes
static LlmManager	Instance
	Singleton instance of the LlmManager.

Properties
bool	IsConnected [get]
	Indicates whether the manager is connected to the LLM server.

Detailed Description

Manages communication with the LLM server, including model registration, loading, unloading, and chat requests. Handles request queuing with parallel execution support.

Member Function Documentation

Chat()

void LlmManager.Chat	(	string	modelID,
		List< Message >	messages,
		Action< ChatResponseDTO >	onComplete,
		Action< string >	onError,
		float	top_p = 0::95f,
		float	temperature = 0::8f,
		int	maxTokens = 4096
	)

Sends a chat request to the LLM server using the specified model and message history.

Parameters

modelID	Unique identifier for the model to use.
messages	List of messages forming the conversation history.
onComplete	Callback on successful response.
onError	Callback on error.
top_p	top_p LLM parameter
temperature	Temperature LLM parameter
maxTokens	Max tokens to generate

Connect()

void LlmManager.Connect

(

Action< bool >

onComplete

)

Connects to the LLM server and unloads all currently loaded models.

Parameters

onComplete

Callback with connection status (true if healthy).

GenericComplete()

void LlmManager.GenericComplete

(

MessageDTO

message

)

Generic callback for handling MessageDTO responses, logs success or error messages.

Parameters

message

The message DTO returned from the server.

Get< T >()

IEnumerator LlmManager.Get< T >	(	string	endpoint,
		Action< T >	onSuccess,
		Action< string >	onError
	)

Sends a GET request to the specified endpoint and deserializes the response to type T.

Template Parameters

T	Type to deserialize the response to.

Parameters

endpoint	API endpoint.
onSuccess	Callback on successful response.
onError	Callback on error.

Returns

Coroutine enumerator.

Type Constraints

T

:

class

LLMQueue()

static bool LlmManager.LLMQueue ( )

static

QueuePostRequest< T, TRequest >()

void LlmManager.QueuePostRequest< T, TRequest >	(	string	endpoint,
		TRequest	data,
		Action< T >	onSuccess,
		Action< string >	onError
	)

Queues a POST request. Execution starts if the number of active requests is below MaxParallelRequests.

Template Parameters

T	Type to deserialize the response to.
TRequest	Type of the request data.

Parameters

endpoint	API endpoint.
data	Request data.
onSuccess	Callback on successful response.
onError	Callback on error.

Type Constraints

T	:	class
TRequest	:	class

Setup()

void LlmManager.Setup

(

string

api

)

Sets up the LLM manager with the specified API base URL.

Parameters

api	Base URL of the LLM server API.

Status()

void LlmManager.Status	(	Action< StatusDTO >	onComplete,
		Action< string >	onError
	)

Gets the status of the LLM server.

Parameters

onComplete	Callback on successful response.
onError	Callback on error.

StatusCommand()

static bool LlmManager.StatusCommand ( )

static

Console command to check the status of the LLM server.

Returns

True if the command was executed.

Member Data Documentation

Instance

LlmManager LlmManager.Instance

static

Singleton instance of the LlmManager.

LogDebug

bool LlmManager.LogDebug = false

Whether log requests and responses.

MaxParallelRequests

int LlmManager.MaxParallelRequests = 4

Maximum number of concurrent POST requests allowed.

Property Documentation

IsConnected

bool LlmManager.IsConnected

get

Indicates whether the manager is connected to the LLM server.

---

The documentation for this class was generated from the following file:

• RaymondMaarloeve/Assets/Scripts/LLM/LlmManager.cs