Mathis/llamactl
1
0
Fork 0
mirror of https://github.com/lordmathis/llamactl.git synced 2025-11-06 00:54:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README and installation guide to clarify Docker support and CUDA configuration

Browse Source
This commit is contained in:
Matúš Námešný
2025-09-28 19:33:12 +02:00
parent bb0176b7f5
commit 169ee422ec
2 changed files with 7 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -110,7 +110,8 @@ docker-compose up llamactl-llamacpp -d
docker-compose up llamactl-vllm -d docker-compose up llamactl-vllm -d
``` ```
**Features:** Full CUDA support, automatic latest release installation, no backend dependencies. **Features:** CUDA support, automatic latest release installation, no backend dependencies.
**Note:** Dockerfiles are configured for CUDA. Adapt base images for other platforms (CPU, ROCm, etc.).
For detailed Docker setup and configuration, see the [Installation Guide](docs/getting-started/installation.md). For detailed Docker setup and configuration, see the [Installation Guide](docs/getting-started/installation.md).

8
docs/getting-started/installation.md
View File

@@ -73,12 +73,14 @@ sudo mv llamactl /usr/local/bin/
### Option 2: Docker ### Option 2: Docker
llamactl provides Dockerfiles for creating Docker images with CUDA support for llama.cpp and vLLM backends. The resulting images include the latest llamactl release with the respective backend pre-installed. llamactl provides Dockerfiles for creating Docker images with backends pre-installed. The resulting images include the latest llamactl release with the respective backend.
**Available Dockerfiles:** **Available Dockerfiles (CUDA):**
- **llamactl with llama.cpp CUDA**: `Dockerfile.llamacpp` (based on `ghcr.io/ggml-org/llama.cpp:server`) - **llamactl with llama.cpp CUDA**: `Dockerfile.llamacpp` (based on `ghcr.io/ggml-org/llama.cpp:server-cuda`)
- **llamactl with vLLM CUDA**: `Dockerfile.vllm` (based on `vllm/vllm-openai:latest`) - **llamactl with vLLM CUDA**: `Dockerfile.vllm` (based on `vllm/vllm-openai:latest`)
**Note:** These Dockerfiles are configured for CUDA. For other platforms (CPU, ROCm, Vulkan, etc.), adapt the base image. For llama.cpp, see available tags at [llama.cpp Docker docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/docker.md). For vLLM, check [vLLM docs](https://docs.vllm.ai/en/v0.6.5/serving/deploying_with_docker.html).
#### Using Docker Compose #### Using Docker Compose
```bash ```bash