mirror of
https://github.com/lordmathis/llamactl.git
synced 2025-11-06 00:54:23 +00:00
Create initial documentation structure
This commit is contained in:
154
docs/getting-started/configuration.md
Normal file
154
docs/getting-started/configuration.md
Normal file
@@ -0,0 +1,154 @@
|
||||
# Configuration
|
||||
|
||||
LlamaCtl can be configured through various methods to suit your needs.
|
||||
|
||||
## Configuration File
|
||||
|
||||
Create a configuration file at `~/.llamactl/config.yaml`:
|
||||
|
||||
```yaml
|
||||
# Server configuration
|
||||
server:
|
||||
host: "0.0.0.0"
|
||||
port: 8080
|
||||
cors_enabled: true
|
||||
|
||||
# Authentication (optional)
|
||||
auth:
|
||||
enabled: false
|
||||
# When enabled, configure your authentication method
|
||||
# jwt_secret: "your-secret-key"
|
||||
|
||||
# Default instance settings
|
||||
defaults:
|
||||
backend: "llamacpp"
|
||||
timeout: 300
|
||||
log_level: "info"
|
||||
|
||||
# Paths
|
||||
paths:
|
||||
models_dir: "/path/to/your/models"
|
||||
logs_dir: "/var/log/llamactl"
|
||||
data_dir: "/var/lib/llamactl"
|
||||
|
||||
# Instance limits
|
||||
limits:
|
||||
max_instances: 10
|
||||
max_memory_per_instance: "8GB"
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
You can also configure LlamaCtl using environment variables:
|
||||
|
||||
```bash
|
||||
# Server settings
|
||||
export LLAMACTL_HOST=0.0.0.0
|
||||
export LLAMACTL_PORT=8080
|
||||
|
||||
# Paths
|
||||
export LLAMACTL_MODELS_DIR=/path/to/models
|
||||
export LLAMACTL_LOGS_DIR=/var/log/llamactl
|
||||
|
||||
# Limits
|
||||
export LLAMACTL_MAX_INSTANCES=5
|
||||
```
|
||||
|
||||
## Command Line Options
|
||||
|
||||
View all available command line options:
|
||||
|
||||
```bash
|
||||
llamactl --help
|
||||
```
|
||||
|
||||
Common options:
|
||||
|
||||
```bash
|
||||
# Specify config file
|
||||
llamactl --config /path/to/config.yaml
|
||||
|
||||
# Set log level
|
||||
llamactl --log-level debug
|
||||
|
||||
# Run on different port
|
||||
llamactl --port 9090
|
||||
```
|
||||
|
||||
## Instance Configuration
|
||||
|
||||
When creating instances, you can specify various options:
|
||||
|
||||
### Basic Options
|
||||
|
||||
- `name`: Unique identifier for the instance
|
||||
- `model_path`: Path to the GGUF model file
|
||||
- `port`: Port for the instance to listen on
|
||||
|
||||
### Advanced Options
|
||||
|
||||
- `threads`: Number of CPU threads to use
|
||||
- `context_size`: Context window size
|
||||
- `batch_size`: Batch size for processing
|
||||
- `gpu_layers`: Number of layers to offload to GPU
|
||||
- `memory_lock`: Lock model in memory
|
||||
- `no_mmap`: Disable memory mapping
|
||||
|
||||
### Example Instance Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "production-model",
|
||||
"model_path": "/models/llama-2-13b-chat.gguf",
|
||||
"port": 8081,
|
||||
"options": {
|
||||
"threads": 8,
|
||||
"context_size": 4096,
|
||||
"batch_size": 512,
|
||||
"gpu_layers": 35,
|
||||
"memory_lock": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Security Configuration
|
||||
|
||||
### Enable Authentication
|
||||
|
||||
To enable authentication, update your config file:
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
jwt_secret: "your-very-secure-secret-key"
|
||||
token_expiry: "24h"
|
||||
```
|
||||
|
||||
### HTTPS Configuration
|
||||
|
||||
For production deployments, configure HTTPS:
|
||||
|
||||
```yaml
|
||||
server:
|
||||
tls:
|
||||
enabled: true
|
||||
cert_file: "/path/to/cert.pem"
|
||||
key_file: "/path/to/key.pem"
|
||||
```
|
||||
|
||||
## Logging Configuration
|
||||
|
||||
Configure logging levels and outputs:
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
level: "info" # debug, info, warn, error
|
||||
format: "json" # json or text
|
||||
output: "/var/log/llamactl/app.log"
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Learn about [Managing Instances](../user-guide/managing-instances.md)
|
||||
- Explore [Advanced Configuration](../advanced/monitoring.md)
|
||||
- Set up [Monitoring](../advanced/monitoring.md)
|
||||
55
docs/getting-started/installation.md
Normal file
55
docs/getting-started/installation.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Installation
|
||||
|
||||
This guide will walk you through installing LlamaCtl on your system.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before installing LlamaCtl, ensure you have:
|
||||
|
||||
- Go 1.19 or later
|
||||
- Git
|
||||
- Sufficient disk space for your models
|
||||
|
||||
## Installation Methods
|
||||
|
||||
### Option 1: Download Binary (Recommended)
|
||||
|
||||
Download the latest release from our [GitHub releases page](https://github.com/lordmathis/llamactl/releases):
|
||||
|
||||
```bash
|
||||
# Download for Linux
|
||||
curl -L https://github.com/lordmathis/llamactl/releases/latest/download/llamactl-linux-amd64 -o llamactl
|
||||
|
||||
# Make executable
|
||||
chmod +x llamactl
|
||||
|
||||
# Move to PATH (optional)
|
||||
sudo mv llamactl /usr/local/bin/
|
||||
```
|
||||
|
||||
### Option 2: Build from Source
|
||||
|
||||
If you prefer to build from source:
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/lordmathis/llamactl.git
|
||||
cd llamactl
|
||||
|
||||
# Build the application
|
||||
go build -o llamactl cmd/server/main.go
|
||||
```
|
||||
|
||||
For detailed build instructions, see the [Building from Source](../development/building.md) guide.
|
||||
|
||||
## Verification
|
||||
|
||||
Verify your installation by checking the version:
|
||||
|
||||
```bash
|
||||
llamactl --version
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that LlamaCtl is installed, continue to the [Quick Start](quick-start.md) guide to get your first instance running!
|
||||
86
docs/getting-started/quick-start.md
Normal file
86
docs/getting-started/quick-start.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Quick Start
|
||||
|
||||
This guide will help you get LlamaCtl up and running in just a few minutes.
|
||||
|
||||
## Step 1: Start LlamaCtl
|
||||
|
||||
Start the LlamaCtl server:
|
||||
|
||||
```bash
|
||||
llamactl
|
||||
```
|
||||
|
||||
By default, LlamaCtl will start on `http://localhost:8080`.
|
||||
|
||||
## Step 2: Access the Web UI
|
||||
|
||||
Open your web browser and navigate to:
|
||||
|
||||
```
|
||||
http://localhost:8080
|
||||
```
|
||||
|
||||
You should see the LlamaCtl web interface.
|
||||
|
||||
## Step 3: Create Your First Instance
|
||||
|
||||
1. Click the "Add Instance" button
|
||||
2. Fill in the instance configuration:
|
||||
- **Name**: Give your instance a descriptive name
|
||||
- **Model Path**: Path to your Llama.cpp model file
|
||||
- **Port**: Port for the instance to run on
|
||||
- **Additional Options**: Any extra Llama.cpp parameters
|
||||
|
||||
3. Click "Create Instance"
|
||||
|
||||
## Step 4: Start Your Instance
|
||||
|
||||
Once created, you can:
|
||||
|
||||
- **Start** the instance by clicking the start button
|
||||
- **Monitor** its status in real-time
|
||||
- **View logs** by clicking the logs button
|
||||
- **Stop** the instance when needed
|
||||
|
||||
## Example Configuration
|
||||
|
||||
Here's a basic example configuration for a Llama 2 model:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "llama2-7b",
|
||||
"model_path": "/path/to/llama-2-7b-chat.gguf",
|
||||
"port": 8081,
|
||||
"options": {
|
||||
"threads": 4,
|
||||
"context_size": 2048
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Using the API
|
||||
|
||||
You can also manage instances via the REST API:
|
||||
|
||||
```bash
|
||||
# List all instances
|
||||
curl http://localhost:8080/api/instances
|
||||
|
||||
# Create a new instance
|
||||
curl -X POST http://localhost:8080/api/instances \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "my-model",
|
||||
"model_path": "/path/to/model.gguf",
|
||||
"port": 8081
|
||||
}'
|
||||
|
||||
# Start an instance
|
||||
curl -X POST http://localhost:8080/api/instances/my-model/start
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Learn more about the [Web UI](../user-guide/web-ui.md)
|
||||
- Explore the [API Reference](../user-guide/api-reference.md)
|
||||
- Configure advanced settings in the [Configuration](configuration.md) guide
|
||||
Reference in New Issue
Block a user