Mathis/llamactl
1
0
Fork 0
mirror of https://github.com/lordmathis/llamactl.git synced 2025-11-06 00:54:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update managing-instances.md with new HuggingFace repository and file examples

Browse Source
This commit is contained in:
Matúš Námešný
2025-09-03 23:04:11 +02:00
parent 3013a343f1
commit ef1a2601fb
1 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/user-guide/managing-instances.md
View File

@@ -37,7 +37,7 @@ Each instance is displayed as a card showing:
2. Enter a unique **Name** for your instance (only required field) 2. Enter a unique **Name** for your instance (only required field)
3. Configure model source (choose one): 3. Configure model source (choose one):
- **Model Path**: Full path to your downloaded GGUF model file - **Model Path**: Full path to your downloaded GGUF model file
- **HuggingFace Repo**: Repository name (e.g., `microsoft/Phi-3-mini-4k-instruct-gguf`) - **HuggingFace Repo**: Repository name (e.g., `unsloth/gemma-3-27b-it-GGUF`)
- **HuggingFace File**: Specific file within the repo (optional, uses default if not specified) - **HuggingFace File**: Specific file within the repo (optional, uses default if not specified)
4. Configure optional instance management settings: 4. Configure optional instance management settings:
- **Auto Restart**: Automatically restart instance on failure - **Auto Restart**: Automatically restart instance on failure
@@ -69,13 +69,13 @@ curl -X POST http://localhost:8080/api/instances/my-instance \
}' }'
# Create instance with HuggingFace model # Create instance with HuggingFace model
curl -X POST http://localhost:8080/api/instances/phi3-mini \ curl -X POST http://localhost:8080/api/instances/gemma-3-27b \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"backend_type": "llama_cpp", "backend_type": "llama_cpp",
"backend_options": { "backend_options": {
"hf_repo": "microsoft/Phi-3-mini-4k-instruct-gguf", "hf_repo": "unsloth/gemma-3-27b-it-GGUF",
"hf_file": "Phi-3-mini-4k-instruct-q4.gguf", "hf_file": "gemma-3-27b-it-GGUF.gguf",
"gpu_layers": 32 "gpu_layers": 32
}, },
"auto_restart": true, "auto_restart": true,
@@ -122,7 +122,7 @@ Modify instance settings:
curl -X PUT http://localhost:8080/api/instances/{name} \ curl -X PUT http://localhost:8080/api/instances/{name} \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"options": { "backend_options": {
"threads": 8, "threads": 8,
"context_size": 4096 "context_size": 4096
} }