llamactl API 1.0¶
llamactl is a control server for managing Llama Server instances.
Backends¶
GET /api/v1/backends/llama-cpp/devices¶
List available devices for llama server
Description
Returns a list of available devices for the llama server
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
GET /api/v1/backends/llama-cpp/help¶
Get help for llama server
Description
Returns the help text for the llama server command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/backends/llama-cpp/parse-command¶
Parse llama-server command
Description
Parses a llama-server command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/backends/llama-cpp/version¶
Get version of llama server
Description
Returns the version of the llama server command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/backends/mlx/parse-command¶
Parse mlx_lm.server command
Description
Parses MLX-LM server command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
POST /api/v1/backends/vllm/parse-command¶
Parse vllm serve command
Description
Parses a vLLM serve command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
Instances¶
GET /api/v1/instances¶
List all instances
Description
Returns a list of all instances managed by the server
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
DELETE /api/v1/instances/{name}¶
Delete an instance
Description
Stops and removes a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 204 No Content
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}¶
Get details of a specific instance
Description
Returns the details of a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}¶
Create and start a new instance
Description
Creates a new instance with the provided configuration options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name | |
options |
body | None | No | Instance configuration options |
Response 201 Created
Response 400 Bad Request
Response 500 Internal Server Error
PUT /api/v1/instances/{name}¶
Update an instance's configuration
Description
Updates the configuration of a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name | |
options |
body | None | No | Instance configuration options |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}/logs¶
Get logs from a specific instance
Description
Returns the logs from a specific instance by name with optional line limit
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
lines |
query | None | No | Number of lines to retrieve (default: all lines) | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}/proxy¶
Proxy requests to a specific instance, does not autostart instance if stopped
Description
Forwards HTTP requests to the llama-server instance running on a specific
port
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Response 503 Service Unavailable
POST /api/v1/instances/{name}/proxy¶
Proxy requests to a specific instance, does not autostart instance if stopped
Description
Forwards HTTP requests to the llama-server instance running on a specific
port
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Response 503 Service Unavailable
POST /api/v1/instances/{name}/restart¶
Restart a running instance
Description
Restarts a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}/start¶
Start a stopped instance
Description
Starts a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}/stop¶
Stop a running instance
Description
Stops a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Nodes¶
GET /api/v1/nodes¶
List all configured nodes
Description
Returns a map of all nodes configured in the server (node name -> node
config)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
GET /api/v1/nodes/{name}¶
Get details of a specific node
Description
Returns the details of a specific node by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Node Name |
Response 200 OK
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
System¶
GET /api/v1/version¶
Get llamactl version
Description
Returns the version of the llamactl command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
Llama.cpp¶
GET /llama-cpp/{name}/¶
Proxy requests to llama.cpp UI for the instance
Description
Proxies requests to the llama.cpp UI for the specified instance
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
query | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/apply-template¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/completion¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/detokenize¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/embeddings¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/infill¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/metrics¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /llama-cpp/{name}/props¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/props¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/reranking¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /llama-cpp/{name}/slots¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/tokenize¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
OpenAI¶
POST /v1/¶
OpenAI-compatible proxy endpoint
Description
Handles all POST requests to /v1/*, routing to the appropriate instance
based on the request body. Requires API key authentication via the
Authorization header.
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /v1/models¶
List instances in OpenAI-compatible format
Description
Returns a list of instances in a format compatible with OpenAI API
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error