llamactl/docs/troubleshooting.md

Troubleshooting

Issues specific to Llamactl deployment and operation.

Configuration Issues

Invalid Configuration

Problem: Invalid configuration preventing startup

Solutions:

1. Use minimal configuration:

   server:
     host: "0.0.0.0"
     port: 8080
   instances:
     port_range: [8000, 9000]
   

2. Check data directory permissions:

   # Ensure data directory is writable (default: ~/.local/share/llamactl)
   mkdir -p ~/.local/share/llamactl/{instances,logs}
   

Instance Management Issues

Instance Fails to Start

Problem: Instance fails to start or immediately stops

Solutions:

1. Check instance logs to see the actual error:

   curl http://localhost:8080/api/v1/instances/{name}/logs
   # Or check log files directly
   tail -f ~/.local/share/llamactl/logs/{instance-name}.log
   

2. Verify backend is installed:

   • llama.cpp: Ensure llama-server is in PATH
   • MLX: Ensure mlx-lm Python package is installed
   • vLLM: Ensure vllm Python package is installed

3. Check model path and format:

   • Use absolute paths to model files
   • Verify model format matches backend (GGUF for llama.cpp, etc.)

4. Verify backend command configuration:

   • Check that the backend command is correctly configured in the global config
   • For virtual environments, specify the full path to the command (e.g., /path/to/venv/bin/mlx_lm.server)
   • See the Configuration Guide for backend configuration details
   • Test the backend directly (see Backend-Specific Issues below)

Backend-Specific Issues

Problem: Model loading, memory, GPU, or performance issues

Most model-specific issues (memory, GPU configuration, performance tuning) are backend-specific and should be resolved by consulting the respective backend documentation:

llama.cpp:

• llama.cpp GitHub
• llama-server README

MLX:

• MLX-LM GitHub
• MLX-LM Server Guide

vLLM:

• vLLM Documentation
• OpenAI Compatible Server
• vllm serve Command

Testing backends directly:

Testing your model and configuration directly with the backend helps determine if the issue is with llamactl or the backend itself:

# llama.cpp
llama-server --model /path/to/model.gguf --port 8081

# MLX
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --port 8081

# vLLM
python -m vllm.entrypoints.openai.api_server --model microsoft/DialoGPT-medium --port 8081

API and Network Issues

CORS Errors

Problem: Web UI shows CORS errors in browser console

Solutions:

1. Configure allowed origins:

   server:
     allowed_origins:
       - "http://localhost:3000"
       - "https://yourdomain.com"
   

Authentication Issues

Problem: API requests failing with authentication errors

Solutions:

1. Disable authentication temporarily:

   auth:
     require_management_auth: false
     require_inference_auth: false
   

2. Configure API keys:

   auth:
     management_keys:
       - "your-management-key"
     inference_keys:
       - "your-inference-key"
   

3. Use correct Authorization header:

   curl -H "Authorization: Bearer your-api-key" \
     http://localhost:8080/api/v1/instances
   

Remote Node Issues

Node Configuration

Problem: Remote instances not appearing or cannot be managed

Solutions:

1. Verify node configuration:

   local_node: "main"  # Must match a key in nodes map
   nodes:
     main:
       address: ""     # Empty for local node
     worker1:
       address: "http://worker1.internal:8080"
       api_key: "secure-key"  # Must match worker1's management key
   

2. Check node name consistency:

   • local_node on each node must match what other nodes call it
   • Node names are case-sensitive

3. Test remote node connectivity:

   curl -H "Authorization: Bearer remote-node-key" \
     http://remote-node:8080/api/v1/instances
   

Debugging and Logs

Viewing Instance Logs

# Get instance logs via API
curl http://localhost:8080/api/v1/instances/{name}/logs

# Or check log files directly
tail -f ~/.local/share/llamactl/logs/{instance-name}.log

Enable Debug Logging

export LLAMACTL_LOG_LEVEL=debug
llamactl

Getting Help

When reporting issues, include:

1. System information:

   llamactl --version
   

2. Configuration file (remove sensitive keys)

3. Relevant log output

4. Steps to reproduce the issue