From 169ee422ec4a3db70b95e6119b2a0bc51e26459f Mon Sep 17 00:00:00 2001 From: LordMathis Date: Sun, 28 Sep 2025 19:33:12 +0200 Subject: [PATCH] Update README and installation guide to clarify Docker support and CUDA configuration --- README.md | 3 ++- docs/getting-started/installation.md | 8 +++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index f11d7f3..59681b9 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,8 @@ docker-compose up llamactl-llamacpp -d docker-compose up llamactl-vllm -d ``` -**Features:** Full CUDA support, automatic latest release installation, no backend dependencies. +**Features:** CUDA support, automatic latest release installation, no backend dependencies. +**Note:** Dockerfiles are configured for CUDA. Adapt base images for other platforms (CPU, ROCm, etc.). For detailed Docker setup and configuration, see the [Installation Guide](docs/getting-started/installation.md). diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 703ab0b..96b9fce 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -73,12 +73,14 @@ sudo mv llamactl /usr/local/bin/ ### Option 2: Docker -llamactl provides Dockerfiles for creating Docker images with CUDA support for llama.cpp and vLLM backends. The resulting images include the latest llamactl release with the respective backend pre-installed. +llamactl provides Dockerfiles for creating Docker images with backends pre-installed. The resulting images include the latest llamactl release with the respective backend. -**Available Dockerfiles:** -- **llamactl with llama.cpp CUDA**: `Dockerfile.llamacpp` (based on `ghcr.io/ggml-org/llama.cpp:server`) +**Available Dockerfiles (CUDA):** +- **llamactl with llama.cpp CUDA**: `Dockerfile.llamacpp` (based on `ghcr.io/ggml-org/llama.cpp:server-cuda`) - **llamactl with vLLM CUDA**: `Dockerfile.vllm` (based on `vllm/vllm-openai:latest`) +**Note:** These Dockerfiles are configured for CUDA. For other platforms (CPU, ROCm, Vulkan, etc.), adapt the base image. For llama.cpp, see available tags at [llama.cpp Docker docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/docker.md). For vLLM, check [vLLM docs](https://docs.vllm.ai/en/v0.6.5/serving/deploying_with_docker.html). + #### Using Docker Compose ```bash