Mathis/llamactl
1
0
Fork 0
mirror of https://github.com/lordmathis/llamactl.git synced 2025-11-06 00:54:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update configration.md with reverse proxy related information

Browse Source
This commit is contained in:
Anuruth Lertpiya
2025-09-29 13:54:15 +00:00
parent e50660c379
commit 03a7a5d139
1 changed files with 18 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/getting-started/configuration.md
View File

@@ -29,6 +29,7 @@ backends:
image: "ghcr.io/ggml-org/llama.cpp:server" image: "ghcr.io/ggml-org/llama.cpp:server"
args: ["run", "--rm", "--network", "host", "--gpus", "all"] args: ["run", "--rm", "--network", "host", "--gpus", "all"]
environment: {} environment: {}
response_headers: {} # Additional response headers to send with responses
vllm: vllm:
command: "vllm" command: "vllm"
@@ -39,11 +40,13 @@ backends:
image: "vllm/vllm-openai:latest" image: "vllm/vllm-openai:latest"
args: ["run", "--rm", "--network", "host", "--gpus", "all", "--shm-size", "1g"] args: ["run", "--rm", "--network", "host", "--gpus", "all", "--shm-size", "1g"]
environment: {} environment: {}
response_headers: {} # Additional response headers to send with responses
mlx: mlx:
command: "mlx_lm.server" command: "mlx_lm.server"
args: [] args: []
environment: {} # Environment variables for the backend process environment: {} # Environment variables for the backend process
response_headers: {} # Additional response headers to send with responses
instances: instances:
port_range: [8000, 9000] # Port range for instances port_range: [8000, 9000] # Port range for instances
@@ -129,7 +132,7 @@ backends:
args: ["serve"] args: ["serve"]
environment: {} # Environment variables for the backend process environment: {} # Environment variables for the backend process
docker: docker:
enabled: false enabled: false # Enable Docker runtime (default: false)
image: "vllm/vllm-openai:latest" image: "vllm/vllm-openai:latest"
args: ["run", "--rm", "--network", "host", "--gpus", "all", "--shm-size", "1g"] args: ["run", "--rm", "--network", "host", "--gpus", "all", "--shm-size", "1g"]
environment: {} environment: {}
@@ -154,7 +157,7 @@ backends:
- `args`: Additional arguments passed to `docker run` - `args`: Additional arguments passed to `docker run`
- `environment`: Environment variables for the container (optional) - `environment`: Environment variables for the container (optional)
> If llamactl is behind an nginx proxy, `X-Accel-Buffering: no` may be required for nginx to properly stream the responses without buffering. > If llamactl is behind an NGINX proxy, `X-Accel-Buffering: no` response header may be required for NGINX to properly stream the responses without buffering.
**Environment Variables:** **Environment Variables:**
@@ -166,7 +169,7 @@ backends:
- `LLAMACTL_LLAMACPP_DOCKER_IMAGE` - Docker image to use - `LLAMACTL_LLAMACPP_DOCKER_IMAGE` - Docker image to use
- `LLAMACTL_LLAMACPP_DOCKER_ARGS` - Space-separated Docker arguments - `LLAMACTL_LLAMACPP_DOCKER_ARGS` - Space-separated Docker arguments
- `LLAMACTL_LLAMACPP_DOCKER_ENV` - Docker environment variables in format "KEY1=value1,KEY2=value2" - `LLAMACTL_LLAMACPP_DOCKER_ENV` - Docker environment variables in format "KEY1=value1,KEY2=value2"
- `LLAMACTL_LLAMACPP_RESPONSE_HEADERS` - Response headers in format "KEY1=value1,KEY2=value2" - `LLAMACTL_LLAMACPP_RESPONSE_HEADERS` - Response headers in format "KEY1=value1;KEY2=value2"
**VLLM Backend:** **VLLM Backend:**
- `LLAMACTL_VLLM_COMMAND` - VLLM executable command - `LLAMACTL_VLLM_COMMAND` - VLLM executable command
@@ -176,11 +179,13 @@ backends:
- `LLAMACTL_VLLM_DOCKER_IMAGE` - Docker image to use - `LLAMACTL_VLLM_DOCKER_IMAGE` - Docker image to use
- `LLAMACTL_VLLM_DOCKER_ARGS` - Space-separated Docker arguments - `LLAMACTL_VLLM_DOCKER_ARGS` - Space-separated Docker arguments
- `LLAMACTL_VLLM_DOCKER_ENV` - Docker environment variables in format "KEY1=value1,KEY2=value2" - `LLAMACTL_VLLM_DOCKER_ENV` - Docker environment variables in format "KEY1=value1,KEY2=value2"
- `LLAMACTL_VLLM_RESPONSE_HEADERS` - Response headers in format "KEY1=value1;KEY2=value2"
**MLX Backend:** **MLX Backend:**
- `LLAMACTL_MLX_COMMAND` - MLX executable command - `LLAMACTL_MLX_COMMAND` - MLX executable command
- `LLAMACTL_MLX_ARGS` - Space-separated default arguments - `LLAMACTL_MLX_ARGS` - Space-separated default arguments
- `LLAMACTL_MLX_ENV` - Environment variables in format "KEY1=value1,KEY2=value2" - `LLAMACTL_MLX_ENV` - Environment variables in format "KEY1=value1,KEY2=value2"
- `LLAMACTL_MLX_RESPONSE_HEADERS` - Response headers in format "KEY1=value1;KEY2=value2"
### Instance Configuration ### Instance Configuration