mirror of
https://github.com/lordmathis/llamactl.git
synced 2025-11-06 00:54:23 +00:00
Deployed 84d994c to dev with MkDocs 1.5.3 and mike 2.0.0
This commit is contained in:
BIN
dev/__pycache__/readme_sync.cpython-311.pyc
Normal file
BIN
dev/__pycache__/readme_sync.cpython-311.pyc
Normal file
Binary file not shown.
121
dev/index.html
121
dev/index.html
@@ -389,6 +389,48 @@
|
|||||||
</span>
|
</span>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
<nav class="md-nav" aria-label="Features">
|
||||||
|
<ul class="md-nav__list">
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#easy-model-management" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🚀 Easy Model Management
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#universal-compatibility" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🔗 Universal Compatibility
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#user-friendly-interface" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🌐 User-Friendly Interface
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#smart-operations" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
⚡ Smart Operations
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
</ul>
|
||||||
|
</nav>
|
||||||
|
|
||||||
</li>
|
</li>
|
||||||
|
|
||||||
<li class="md-nav__item">
|
<li class="md-nav__item">
|
||||||
@@ -692,6 +734,48 @@
|
|||||||
</span>
|
</span>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
<nav class="md-nav" aria-label="Features">
|
||||||
|
<ul class="md-nav__list">
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#easy-model-management" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🚀 Easy Model Management
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#universal-compatibility" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🔗 Universal Compatibility
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#user-friendly-interface" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
🌐 User-Friendly Interface
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
<li class="md-nav__item">
|
||||||
|
<a href="#smart-operations" class="md-nav__link">
|
||||||
|
<span class="md-ellipsis">
|
||||||
|
⚡ Smart Operations
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</li>
|
||||||
|
|
||||||
|
</ul>
|
||||||
|
</nav>
|
||||||
|
|
||||||
</li>
|
</li>
|
||||||
|
|
||||||
<li class="md-nav__item">
|
<li class="md-nav__item">
|
||||||
@@ -740,20 +824,33 @@
|
|||||||
|
|
||||||
|
|
||||||
<h1 id="llamactl-documentation">Llamactl Documentation<a class="headerlink" href="#llamactl-documentation" title="Permanent link">¶</a></h1>
|
<h1 id="llamactl-documentation">Llamactl Documentation<a class="headerlink" href="#llamactl-documentation" title="Permanent link">¶</a></h1>
|
||||||
<p>Welcome to the Llamactl documentation! <strong>Management server and proxy for multiple llama.cpp and MLX instances with OpenAI-compatible API routing.</strong></p>
|
<p>Welcome to the Llamactl documentation! </p>
|
||||||
<p><img alt="Dashboard Screenshot" src="images/dashboard.png" /></p>
|
<p><img alt="Dashboard Screenshot" src="images/dashboard.png" /></p>
|
||||||
<h2 id="what-is-llamactl">What is Llamactl?<a class="headerlink" href="#what-is-llamactl" title="Permanent link">¶</a></h2>
|
<h2 id="what-is-llamactl">What is Llamactl?<a class="headerlink" href="#what-is-llamactl" title="Permanent link">¶</a></h2>
|
||||||
<p>Llamactl is designed to simplify the deployment and management of llama-server and MLX instances. It provides a modern solution for running multiple large language models with centralized management and multi-backend support.</p>
|
<p><strong>Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.</strong></p>
|
||||||
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">¶</a></h2>
|
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">¶</a></h2>
|
||||||
<p>🚀 <strong>Multiple Model Serving</strong>: Run different models simultaneously (7B for speed, 70B for quality)
|
<h3 id="easy-model-management">🚀 Easy Model Management<a class="headerlink" href="#easy-model-management" title="Permanent link">¶</a></h3>
|
||||||
🔗 <strong>OpenAI API Compatible</strong>: Drop-in replacement - route requests by model name
|
<ul>
|
||||||
🍎 <strong>Multi-Backend Support</strong>: Native support for both llama.cpp and MLX (Apple Silicon optimized)
|
<li><strong>Multiple Model Serving</strong>: Run different models simultaneously (7B for speed, 70B for quality) </li>
|
||||||
🌐 <strong>Web Dashboard</strong>: Modern React UI for visual management (unlike CLI-only tools)
|
<li><strong>On-Demand Instance Start</strong>: Automatically launch instances upon receiving API requests </li>
|
||||||
🔐 <strong>API Key Authentication</strong>: Separate keys for management vs inference access
|
<li><strong>State Persistence</strong>: Ensure instances remain intact across server restarts </li>
|
||||||
📊 <strong>Instance Monitoring</strong>: Health checks, auto-restart, log management
|
</ul>
|
||||||
⚡ <strong>Smart Resource Management</strong>: Idle timeout, LRU eviction, and configurable instance limits
|
<h3 id="universal-compatibility">🔗 Universal Compatibility<a class="headerlink" href="#universal-compatibility" title="Permanent link">¶</a></h3>
|
||||||
💡 <strong>On-Demand Instance Start</strong>: Automatically launch instances upon receiving OpenAI-compatible API requests
|
<ul>
|
||||||
💾 <strong>State Persistence</strong>: Ensure instances remain intact across server restarts </p>
|
<li><strong>OpenAI API Compatible</strong>: Drop-in replacement - route requests by instance name </li>
|
||||||
|
<li><strong>Multi-Backend Support</strong>: Native support for llama.cpp, MLX (Apple Silicon optimized), and vLLM </li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="user-friendly-interface">🌐 User-Friendly Interface<a class="headerlink" href="#user-friendly-interface" title="Permanent link">¶</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Web Dashboard</strong>: Modern React UI for visual management (unlike CLI-only tools) </li>
|
||||||
|
<li><strong>API Key Authentication</strong>: Separate keys for management vs inference access </li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="smart-operations">⚡ Smart Operations<a class="headerlink" href="#smart-operations" title="Permanent link">¶</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Instance Monitoring</strong>: Health checks, auto-restart, log management </li>
|
||||||
|
<li><strong>Smart Resource Management</strong>: Idle timeout, LRU eviction, and configurable instance limits </li>
|
||||||
|
</ul>
|
||||||
|
<p><img alt="Dashboard Screenshot" src="images/dashboard.png" /> </p>
|
||||||
<h2 id="quick-links">Quick Links<a class="headerlink" href="#quick-links" title="Permanent link">¶</a></h2>
|
<h2 id="quick-links">Quick Links<a class="headerlink" href="#quick-links" title="Permanent link">¶</a></h2>
|
||||||
<ul>
|
<ul>
|
||||||
<li><a href="getting-started/installation/">Installation Guide</a> - Get Llamactl up and running</li>
|
<li><a href="getting-started/installation/">Installation Guide</a> - Get Llamactl up and running</li>
|
||||||
@@ -791,7 +888,7 @@
|
|||||||
<span class="md-icon" title="Last update">
|
<span class="md-icon" title="Last update">
|
||||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1-2.1-2M12.5 7v5.2l4 2.4-1 1L11 13V7h1.5M11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2v1.8Z"/></svg>
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1-2.1-2M12.5 7v5.2l4 2.4-1 1L11 13V7h1.5M11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2v1.8Z"/></svg>
|
||||||
</span>
|
</span>
|
||||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 18, 2025</span>
|
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 22, 2025</span>
|
||||||
</span>
|
</span>
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
62
dev/readme_sync.py
Normal file
62
dev/readme_sync.py
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
"""
|
||||||
|
MkDocs hook to sync content from README.md to docs/index.md
|
||||||
|
"""
|
||||||
|
import re
|
||||||
|
import os
|
||||||
|
|
||||||
|
|
||||||
|
def on_page_markdown(markdown, page, config, **kwargs):
|
||||||
|
"""Process markdown content before rendering"""
|
||||||
|
# Only process the index.md file
|
||||||
|
if page.file.src_path != 'index.md':
|
||||||
|
return markdown
|
||||||
|
|
||||||
|
# Get the path to README.md (relative to mkdocs.yml)
|
||||||
|
readme_path = os.path.join(os.path.dirname(config['config_file_path']), 'README.md')
|
||||||
|
|
||||||
|
if not os.path.exists(readme_path):
|
||||||
|
print(f"Warning: README.md not found at {readme_path}")
|
||||||
|
return markdown
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(readme_path, 'r', encoding='utf-8') as f:
|
||||||
|
readme_content = f.read()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Error reading README.md: {e}")
|
||||||
|
return markdown
|
||||||
|
|
||||||
|
# Extract headline (the text in bold after the title)
|
||||||
|
headline_match = re.search(r'\*\*(.*?)\*\*', readme_content)
|
||||||
|
headline = headline_match.group(1) if headline_match else 'Management server for llama.cpp and MLX instances'
|
||||||
|
|
||||||
|
# Extract features section - everything between ## Features and the next ## heading
|
||||||
|
features_match = re.search(r'## Features\n(.*?)(?=\n## |\Z)', readme_content, re.DOTALL)
|
||||||
|
if features_match:
|
||||||
|
features_content = features_match.group(1).strip()
|
||||||
|
# Just add line breaks at the end of each line for proper MkDocs rendering
|
||||||
|
features_with_breaks = add_line_breaks(features_content)
|
||||||
|
else:
|
||||||
|
features_with_breaks = "Features content not found in README.md"
|
||||||
|
|
||||||
|
# Replace placeholders in the markdown
|
||||||
|
markdown = markdown.replace('{{HEADLINE}}', headline)
|
||||||
|
markdown = markdown.replace('{{FEATURES}}', features_with_breaks)
|
||||||
|
|
||||||
|
# Fix image paths: convert docs/images/ to images/ for MkDocs
|
||||||
|
markdown = re.sub(r'docs/images/', 'images/', markdown)
|
||||||
|
|
||||||
|
return markdown
|
||||||
|
|
||||||
|
|
||||||
|
def add_line_breaks(content):
|
||||||
|
"""Add two spaces at the end of each line for proper MkDocs line breaks"""
|
||||||
|
lines = content.split('\n')
|
||||||
|
processed_lines = []
|
||||||
|
|
||||||
|
for line in lines:
|
||||||
|
if line.strip(): # Only add spaces to non-empty lines
|
||||||
|
processed_lines.append(line.rstrip() + ' ')
|
||||||
|
else:
|
||||||
|
processed_lines.append(line)
|
||||||
|
|
||||||
|
return '\n'.join(processed_lines)
|
||||||
File diff suppressed because one or more lines are too long
Binary file not shown.
@@ -1571,7 +1571,7 @@
|
|||||||
</code></pre></div></p>
|
</code></pre></div></p>
|
||||||
<p>The server routes requests to the appropriate instance based on the <code>model</code> field in the request body. Instances with on-demand starting enabled will be automatically started if not running. For configuration details, see <a href="../managing-instances/">Managing Instances</a>.</p>
|
<p>The server routes requests to the appropriate instance based on the <code>model</code> field in the request body. Instances with on-demand starting enabled will be automatically started if not running. For configuration details, see <a href="../managing-instances/">Managing Instances</a>.</p>
|
||||||
<p><strong>Error Responses:</strong>
|
<p><strong>Error Responses:</strong>
|
||||||
- <code>400 Bad Request</code>: Invalid request body or missing model name
|
- <code>400 Bad Request</code>: Invalid request body or missing instance name
|
||||||
- <code>503 Service Unavailable</code>: Instance is not running and on-demand start is disabled
|
- <code>503 Service Unavailable</code>: Instance is not running and on-demand start is disabled
|
||||||
- <code>409 Conflict</code>: Cannot start instance due to maximum instances limit</p>
|
- <code>409 Conflict</code>: Cannot start instance due to maximum instances limit</p>
|
||||||
<h2 id="instance-status-values">Instance Status Values<a class="headerlink" href="#instance-status-values" title="Permanent link">¶</a></h2>
|
<h2 id="instance-status-values">Instance Status Values<a class="headerlink" href="#instance-status-values" title="Permanent link">¶</a></h2>
|
||||||
@@ -1739,7 +1739,7 @@
|
|||||||
<span class="md-icon" title="Last update">
|
<span class="md-icon" title="Last update">
|
||||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1-2.1-2M12.5 7v5.2l4 2.4-1 1L11 13V7h1.5M11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2v1.8Z"/></svg>
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1-2.1-2M12.5 7v5.2l4 2.4-1 1L11 13V7h1.5M11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2v1.8Z"/></svg>
|
||||||
</span>
|
</span>
|
||||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 21, 2025</span>
|
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">September 22, 2025</span>
|
||||||
</span>
|
</span>
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user