Mathis/llamactl
1
0
Fork 0
mirror of https://github.com/lordmathis/llamactl.git synced 2025-11-06 00:54:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Refactor features section in README for improved clarity and organization

Browse Source
This commit is contained in:
Matúš Námešný
2025-09-18 20:14:03 +02:00
parent 8006dd3841
commit 84c3453281
1 changed files with 17 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@@ -4,17 +4,24 @@
**Management server and proxy for multiple llama.cpp and MLX instances with OpenAI-compatible API routing.** **Management server and proxy for multiple llama.cpp and MLX instances with OpenAI-compatible API routing.**
## Why llamactl? ## Features
🚀 **Multiple Model Serving**: Run different models simultaneously (7B for speed, 70B for quality) ### 🚀 Easy Model Management
🔗 **OpenAI API Compatible**: Drop-in replacement - route requests by model name - **Multiple Model Serving**: Run different models simultaneously (7B for speed, 70B for quality)
🍎 **Multi-Backend Support**: Native support for both llama.cpp and MLX (Apple Silicon optimized) - **On-Demand Instance Start**: Automatically launch instances upon receiving API requests
🌐 **Web Dashboard**: Modern React UI for visual management (unlike CLI-only tools) - **State Persistence**: Ensure instances remain intact across server restarts
🔐 **API Key Authentication**: Separate keys for management vs inference access
📊 **Instance Monitoring**: Health checks, auto-restart, log management ### 🔗 Universal Compatibility
⚡ **Smart Resource Management**: Idle timeout, LRU eviction, and configurable instance limits - **OpenAI API Compatible**: Drop-in replacement - route requests by model name
💡 **On-Demand Instance Start**: Automatically launch instances upon receiving OpenAI-compatible API requests - **Multi-Backend Support**: Native support for both llama.cpp and MLX (Apple Silicon optimized)
💾 **State Persistence**: Ensure instances remain intact across server restarts
### 🌐 User-Friendly Interface
- **Web Dashboard**: Modern React UI for visual management (unlike CLI-only tools)
- **API Key Authentication**: Separate keys for management vs inference access
### ⚡ Smart Operations
- **Instance Monitoring**: Health checks, auto-restart, log management
- **Smart Resource Management**: Idle timeout, LRU eviction, and configurable instance limits
![Dashboard Screenshot](docs/images/dashboard.png) ![Dashboard Screenshot](docs/images/dashboard.png)