mirror of
https://github.com/lordmathis/llamactl.git
synced 2025-11-06 00:54:23 +00:00
Deployed 84d994c to dev with MkDocs 1.5.3 and mike 2.0.0
This commit is contained in:
lordmathis
121
dev/index.html
121
dev/index.html
@@ -389,6 +389,48 @@
|
||||
</span>
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Features">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#easy-model-management" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🚀 Easy Model Management
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#universal-compatibility" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🔗 Universal Compatibility
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#user-friendly-interface" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🌐 User-Friendly Interface
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#smart-operations" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
⚡ Smart Operations
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -692,6 +734,48 @@
|
||||
</span>
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Features">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#easy-model-management" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🚀 Easy Model Management
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#universal-compatibility" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🔗 Universal Compatibility
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#user-friendly-interface" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
🌐 User-Friendly Interface
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#smart-operations" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
⚡ Smart Operations
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -740,20 +824,33 @@
|
||||
|
||||
|
||||
<h1 id="llamactl-documentation">Llamactl Documentation<a class="headerlink" href="#llamactl-documentation" title="Permanent link">¶</a></h1>
|
||||
<p>Welcome to the Llamactl documentation! <strong>Management server and proxy for multiple llama.cpp and MLX instances with OpenAI-compatible API routing.</strong></p>
|
||||
<p>Welcome to the Llamactl documentation! </p>
|
||||
<p><img alt="Dashboard Screenshot" src="images/dashboard.png" /></p>
|
||||
<h2 id="what-is-llamactl">What is Llamactl?<a class="headerlink" href="#what-is-llamactl" title="Permanent link">¶</a></h2>
|
||||
<p>Llamactl is designed to simplify the deployment and management of llama-server and MLX instances. It provides a modern solution for running multiple large language models with centralized management and multi-backend support.</p>
|
||||
<p><strong>Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.</strong></p>
|
||||
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">¶</a></h2>
|
||||
<p>🚀 <strong>Multiple Model Serving</strong>: Run different models simultaneously (7B for speed, 70B for quality)
|
||||
🔗 <strong>OpenAI API Compatible</strong>: Drop-in replacement - route requests by model name
|
||||
🍎 <strong>Multi-Backend Support</strong>: Native support for both llama.cpp and MLX (Apple Silicon optimized)
|
||||
🌐 <strong>Web Dashboard</strong>: Modern React UI for visual management (unlike CLI-only tools)
|
||||
🔐 <strong>API Key Authentication</strong>: Separate keys for management vs inference access
|
||||
📊 <strong>Instance Monitoring</strong>: Health checks, auto-restart, log management
|
||||
⚡ <strong>Smart Resource Management</strong>: Idle timeout, LRU eviction, and configurable instance limits
|
||||
💡 <strong>On-Demand Instance Start</strong>: Automatically launch instances upon receiving OpenAI-compatible API requests
|
||||
💾 <strong>State Persistence</strong>: Ensure instances remain intact across server restarts </p>
|
||||
<h3 id="easy-model-management">🚀 Easy Model Management<a class="headerlink" href="#easy-model-management" title="Permanent link">¶</a></h3>
|
||||
<ul>
|
||||
<li><strong>Multiple Model Serving</strong>: Run different models simultaneously (7B for speed, 70B for quality) </li>
|
||||
<li><strong>On-Demand Instance Start</strong>: Automatically launch instances upon receiving API requests </li>
|
||||
<li><strong>State Persistence</strong>: Ensure instances remain intact across server restarts </li>
|
||||
</ul>
|
||||
<h3 id="universal-compatibility">🔗 Universal Compatibility<a class="headerlink" href="#universal-compatibility" title="Permanent link">¶</a></h3>
|
||||
<ul>
|
||||
<li><strong>OpenAI API Compatible</strong>: Drop-in replacement - route requests by instance name </li>
|
||||
<li><strong>Multi-Backend Support</strong>: Native support for llama.cpp, MLX (Apple Silicon optimized), and vLLM </li>
|
||||
</ul>
|
||||
<h3 id="user-friendly-interface">🌐 User-Friendly Interface<a class="headerlink" href="#user-friendly-interface" title="Permanent link">¶</a></h3>
|
||||
<ul>
|
||||
<li><strong>Web Dashboard</strong>: Modern React UI for visual management (unlike CLI-only tools) </li>
|
||||
<li><strong>API Key Authentication</strong>: Separate keys for management vs inference access </li>
|
||||
</ul>
|
||||
<h3 id="smart-operations">⚡ Smart Operations<a class="headerlink" href="#smart-operations" title="Permanent link">¶</a></h3>
|
||||
<ul>
|
||||
<li><strong>Instance Monitoring</strong>: Health checks, auto-restart, log management </li>
|
||||
<li><strong>Smart Resource Management</strong>: Idle timeout, LRU eviction, and configurable instance limits </li>
|
||||
</ul>
|
||||
<p><img alt="Dashboard Screenshot" src="images/dashboard.png" /> </p>
|
||||
<h2 id="quick-links">Quick Links<a class="headerlink" href="#quick-links" title="Permanent link">¶</a></h2>
|
||||
<ul>
|
||||
<li><a href="getting-started/installation/">Installation Guide</a> - Get Llamactl up and running</li>
|
||||
@@ -791,7 +888,7 @@
|
||||
<span class="md-icon" title="Last update">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1-2.1-2M12.5 7v5.2l4 2.4-1 1L11 13V7h1.5M11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2v1.8Z"/></svg>
|
||||
</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 18, 2025</span>
|
||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 22, 2025</span>
|
||||
</span>
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user