llamactl API 1.0¶
+llamactl is a control server for managing Llama Server instances.
++
Backends¶
++ +
GET /api/v1/backends/llama-cpp/devices¶
+List available devices for llama server
+Description
+Returns a list of available devices for the llama server
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/backends/llama-cpp/help¶
+Get help for llama server
+Description
+Returns the help text for the llama server command
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /api/v1/backends/llama-cpp/parse-command¶
+Parse llama-server command
+Description
+Parses a llama-server command string into instance options
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
request |
+ body | +None | ++ | No | +Command to parse | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/backends/llama-cpp/version¶
+Get version of llama server
+Description
+Returns the version of the llama server command
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /api/v1/backends/mlx/parse-command¶
+Parse mlx_lm.server command
+Description
+Parses MLX-LM server command string into instance options
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
request |
+ body | +None | ++ | No | +Command to parse | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ +
POST /api/v1/backends/vllm/parse-command¶
+Parse vllm serve command
+Description
+Parses a vLLM serve command string into instance options
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
request |
+ body | +None | ++ | No | +Command to parse | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ +Instances¶
++ +
GET /api/v1/instances¶
+List all instances
+Description
+Returns a list of all instances managed by the server
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ +
DELETE /api/v1/instances/{name}¶
+Delete an instance
+Description
+Stops and removes a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 204 No Content +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/instances/{name}¶
+Get details of a specific instance
+Description
+Returns the details of a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /api/v1/instances/{name}¶
+Create and start a new instance
+Description
+Creates a new instance with the provided configuration options
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
options |
+ body | +None | ++ | No | +Instance configuration options | +
+ Response 201 Created +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
PUT /api/v1/instances/{name}¶
+Update an instance's configuration
+Description
+Updates the configuration of a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
options |
+ body | +None | ++ | No | +Instance configuration options | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/instances/{name}/logs¶
+Get logs from a specific instance
+Description
+Returns the logs from a specific instance by name with optional line limit
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
lines |
+ query | +None | ++ | No | +Number of lines to retrieve (default: all lines) | +
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/instances/{name}/proxy¶
+Proxy requests to a specific instance, does not autostart instance if stopped
+Description
+Forwards HTTP requests to the llama-server instance running on a specific
+port
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ Response 503 Service Unavailable +
+ ++ +
POST /api/v1/instances/{name}/proxy¶
+Proxy requests to a specific instance, does not autostart instance if stopped
+Description
+Forwards HTTP requests to the llama-server instance running on a specific
+port
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ Response 503 Service Unavailable +
+ ++ +
POST /api/v1/instances/{name}/restart¶
+Restart a running instance
+Description
+Restarts a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /api/v1/instances/{name}/start¶
+Start a stopped instance
+Description
+Starts a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /api/v1/instances/{name}/stop¶
+Stop a running instance
+Description
+Stops a specific instance by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ +Nodes¶
++ +
GET /api/v1/nodes¶
+List all configured nodes
+Description
+Returns a map of all nodes configured in the server (node name -> node
+config)
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /api/v1/nodes/{name}¶
+Get details of a specific node
+Description
+Returns the details of a specific node by name
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Node Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 404 Not Found +
+ ++ Response 500 Internal Server Error +
+ +System¶
++ +
GET /api/v1/version¶
+Get llamactl version
+Description
+Returns the version of the llamactl command
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ +Llama.cpp¶
++ +
GET /llama-cpp/{name}/¶
+Proxy requests to llama.cpp UI for the instance
+Description
+Proxies requests to the llama.cpp UI for the specified instance
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ query | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/apply-template¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/completion¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/detokenize¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/embeddings¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/infill¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/metrics¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /llama-cpp/{name}/props¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/props¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/reranking¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /llama-cpp/{name}/slots¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
POST /llama-cpp/{name}/tokenize¶
+Proxy requests to llama.cpp server instance
+Description
+Proxies requests to the specified llama.cpp server instance, starting it on-
+demand if configured
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
name |
+ path | +None | ++ | No | +Instance Name | +
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ +OpenAI¶
++ +
POST /v1/¶
+OpenAI-compatible proxy endpoint
+Description
+Handles all POST requests to /v1/*, routing to the appropriate instance
+based on the request body. Requires API key authentication via the
+Authorization header.
Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 400 Bad Request +
+ ++ Response 500 Internal Server Error +
+ ++ +
GET /v1/models¶
+List instances in OpenAI-compatible format
+Description
+Returns a list of instances in a format compatible with OpenAI API
+Input parameters
+| Parameter | +In | +Type | +Default | +Nullable | +Description | +
|---|---|---|---|---|---|
ApiKeyAuth |
+ header | +string | +N/A | +No | ++ |
+ Response 200 OK +
+ ++ Response 500 Internal Server Error +
+ ++ + + + + + + + + + + + + + + + + + + + +