0188f82306
Implement remote instance creation and deletion in instance manager
2025-10-01 22:05:18 +02:00
e0f176de10
Enhance instance manager to support remote instance management and update related tests
2025-10-01 20:25:06 +02:00
2759be65a5
Add remote instance management functionality and configuration support
2025-09-30 21:09:05 +02:00
Anuruth Lertpiya
5906d89f8d
Added support for serving behind a reverse proxy
...
- Added support for specifying response headers for each backend
- Allowing users to set `X-Accel-Buffering: no` to disable buffering for streaming responses in nginx
- Updated `configuration.md` to document the new configuration options
- Modified Vite config to build with `base: "./"`, making assets be accessed via relative paths
- Updated API_BASE to use `document.baseURI`, allowing API calls to be made relative to the base path
2025-09-29 12:43:10 +00:00
0ecd55c354
Start with host environment for instances
2025-09-28 21:37:48 +02:00
808092decf
Initialize context in Start method for command execution
2025-09-28 20:51:11 +02:00
fa9335663a
Parse backend env vars from env vars
2025-09-28 15:22:01 +02:00
c984d95723
Add environment variable support to instance options and command building
2025-09-28 14:42:10 +02:00
50e1355205
Add environment field to BackendSettings for improved configuration
2025-09-28 14:42:10 +02:00
Anuruth Lertpiya
f496a28f04
fix: InstancesDir and LogsDir not being relative path to DataDir when not set
2025-09-27 18:14:25 +00:00
Anuruth Lertpiya
3a979da815
fix: llamactl reads config file per documentation
...
- Added logging to track config file reading operations
- llamactl now properly reads config files from the expected locations ("llamactl.yaml" and "config.yaml" under current directory)
2025-09-27 17:03:54 +00:00
2cd9d374a7
Add Docker badge to UI
2025-09-25 23:04:24 +02:00
031d6c7017
Update Docker command arguments for llama-server and vllm with volume mounts
2025-09-25 22:51:51 +02:00
282344af23
Fix docker command args building
2025-09-25 22:51:40 +02:00
bc9e0535c3
Refactor command building and argument handling
2025-09-25 22:05:46 +02:00
ba0f877185
Fix tests
2025-09-24 21:35:44 +02:00
840a7bc650
Add Docker command handling for backend options and refactor command building
2025-09-24 21:34:54 +02:00
76ac93bedc
Implement Docker command handling for Llama, MLX, and vLLM backends
2025-09-24 21:31:58 +02:00
72d2a601c8
Update Docker args in LoadConfig and tests to include 'run --rm' prefix
2025-09-24 21:27:51 +02:00
9a56660f68
Refactor backend configuration to use structured settings and update environment variable handling
2025-09-24 20:31:20 +02:00
30e40ecd30
Refactor API endpoints to use /backends/llama-cpp path and update related documentation
2025-09-23 21:27:58 +02:00
46622d2107
Update documentation and add README synchronization
2025-09-22 22:37:53 +02:00
184d6df1bc
Fix vllm command parsing
2025-09-22 21:25:50 +02:00
313666ea17
Fix missing vllm proxy setup
2025-09-22 20:51:00 +02:00
c3ca5b95f7
Update BuildCommandArgs to use positional argument for model and adjust tests accordingly
2025-09-22 20:32:03 +02:00
7eb59aa7e0
Remove unused JSON unmarshal test and clean up command argument checks
2025-09-19 20:46:25 +02:00
64842e74b0
Refactor command parsing and building
2025-09-19 20:23:25 +02:00
34a949d22e
Refactor command argument building and parsing
2025-09-19 19:59:46 +02:00
ec5485bd0e
Refactor command argument building across backends
2025-09-19 19:46:54 +02:00
9eecb37aec
Refactor MLX and VLLM server options parsing and args building
2025-09-19 19:39:36 +02:00
c7136d5206
Refactor command parsing logic across backends to utilize a unified CommandParserConfig structure
2025-09-19 18:36:23 +02:00
4df02a6519
Initial vLLM backend support
2025-09-19 18:05:12 +02:00
6a580667ed
Remove LlamaExecutable checks from default and file loading tests
2025-09-18 20:30:26 +02:00
2a20817078
Remove redundant LlamaExecutable field from instance configuration in tests
2025-09-18 20:29:04 +02:00
5121f0e302
Remove PythonPath references from MlxServerOptions and related configurations
2025-09-17 21:59:55 +02:00
cc5d8acd92
Refactor instance and manager tests to use BackendConfig for LlamaExecutable and MLXLMExecutable
2025-09-16 21:45:50 +02:00
154b754aff
Add MLX command parsing and routing support
2025-09-16 21:39:08 +02:00
63fea02d66
Add MLX backend support in CreateInstanceOptions and validation
2025-09-16 21:38:33 +02:00
468688cdbc
Pass backend options to instances
2025-09-16 21:37:48 +02:00
988c4aca40
Add MLX backend config options
2025-09-16 21:14:19 +02:00
1b5934303b
Enhance command parsing in ParseLlamaCommand and improve error handling in ParseCommandRequest
2025-09-15 22:12:56 +02:00
e7b06341c3
Enhance command parsing in ParseLlamaCommand
2025-09-15 21:29:46 +02:00
323056096c
Implement llama-server command parsing and add UI components for command input
2025-09-15 21:04:14 +02:00
d697f83b46
Update GetProxy method to use BackendTypeLlamaCpp constant for backend type
2025-09-02 21:56:38 +02:00
712d28ea42
Remove port marking logic from CreateInstance method
2025-09-02 21:56:25 +02:00
d9542ba117
Refactor instance management to support backend types and options
2025-09-01 21:59:18 +02:00
9579930a6a
Simplify LRU eviction tests
2025-08-31 11:46:16 +02:00
447f441fd0
Move LRU eviction to timeout.go
2025-08-31 11:42:32 +02:00
27012b6de6
Split manager tests into multiple test files
2025-08-31 11:39:44 +02:00
905e685107
Add LRU eviction tests for instance management
2025-08-31 11:30:57 +02:00
