5afc22924f
Refactor Status struct
2025-10-16 20:15:22 +02:00
e0ec00d141
Remove rendundant instance prefix from status
2025-10-16 19:40:03 +02:00
80ca0cbd4f
Rename Process to Instance
2025-10-16 19:38:44 +02:00
964c6345ef
Refactor backend host/port retrieval and remove redundant code for health checks
2025-10-14 22:16:26 +02:00
92a76bc84b
Move proxy to separate struct
2025-10-14 22:01:09 +02:00
02909c5153
Remove redundant instance prefix from logger
2025-10-14 19:46:43 +02:00
ef3478e2a3
Move logging to separate struct
2025-10-14 19:32:15 +02:00
b965b77c18
Prevent remote instances from using local proxy in GetProxy method
2025-10-09 20:24:54 +02:00
8d9b0c0621
Initialize timeProvider and logger in UnmarshalJSON for Process
2025-10-09 18:56:12 +02:00
2759be65a5
Add remote instance management functionality and configuration support
2025-09-30 21:09:05 +02:00
Anuruth Lertpiya
5906d89f8d
Added support for serving behind a reverse proxy
...
- Added support for specifying response headers for each backend
- Allowing users to set `X-Accel-Buffering: no` to disable buffering for streaming responses in nginx
- Updated `configuration.md` to document the new configuration options
- Modified Vite config to build with `base: "./"`, making assets be accessed via relative paths
- Updated API_BASE to use `document.baseURI`, allowing API calls to be made relative to the base path
2025-09-29 12:43:10 +00:00
0ecd55c354
Start with host environment for instances
2025-09-28 21:37:48 +02:00
808092decf
Initialize context in Start method for command execution
2025-09-28 20:51:11 +02:00
c984d95723
Add environment variable support to instance options and command building
2025-09-28 14:42:10 +02:00
2cd9d374a7
Add Docker badge to UI
2025-09-25 23:04:24 +02:00
282344af23
Fix docker command args building
2025-09-25 22:51:40 +02:00
bc9e0535c3
Refactor command building and argument handling
2025-09-25 22:05:46 +02:00
ba0f877185
Fix tests
2025-09-24 21:35:44 +02:00
840a7bc650
Add Docker command handling for backend options and refactor command building
2025-09-24 21:34:54 +02:00
76ac93bedc
Implement Docker command handling for Llama, MLX, and vLLM backends
2025-09-24 21:31:58 +02:00
313666ea17
Fix missing vllm proxy setup
2025-09-22 20:51:00 +02:00
4df02a6519
Initial vLLM backend support
2025-09-19 18:05:12 +02:00
cc5d8acd92
Refactor instance and manager tests to use BackendConfig for LlamaExecutable and MLXLMExecutable
2025-09-16 21:45:50 +02:00
63fea02d66
Add MLX backend support in CreateInstanceOptions and validation
2025-09-16 21:38:33 +02:00
468688cdbc
Pass backend options to instances
2025-09-16 21:37:48 +02:00
d697f83b46
Update GetProxy method to use BackendTypeLlamaCpp constant for backend type
2025-09-02 21:56:38 +02:00
d9542ba117
Refactor instance management to support backend types and options
2025-09-01 21:59:18 +02:00
905e685107
Add LRU eviction tests for instance management
2025-08-31 11:30:57 +02:00
c1fa0faf4b
Add LastRequestTime method and LRU eviction logic for instance management
2025-08-30 23:59:37 +02:00
e319731239
Remove unnecessary read locks from GetStatus and IsRunning methods
2025-08-28 19:19:28 +02:00
b698c1d0ea
Remove locks from SetStatus
2025-08-28 19:08:20 +02:00
227ca7927a
Refactor SetStatus method to capture onStatusChange callback reference before unlocking mutex
2025-08-28 18:59:26 +02:00
ae37055331
Add onStatusChange callback to instance management for status updates
2025-08-27 20:54:26 +02:00
b41ebdc604
Set instance status to Failed when restart conditions are not met
2025-08-27 19:47:36 +02:00
1443746add
Refactor instance status management: replace Running boolean with InstanceStatus enum and update related methods
2025-08-27 19:44:38 +02:00
615c2ac54e
Add MaxRunningInstances to InstancesConfig and implement IsRunning method
2025-08-27 18:42:34 +02:00
1939b45312
Refactor WaitForHealthy method to use direct health check URL and simplify health check logic
2025-08-20 15:58:08 +02:00
287a5e0817
Implement WaitForHealthy method and enhance OpenAIProxy to support on-demand instance start
2025-08-20 14:19:12 +02:00
7b4adfa0cd
Add DefaultOnDemandStart configuration and update instance options
2025-08-20 13:50:43 +02:00
492c3ff270
Remove redundant timeout tests and improve test coverage for instance timeout validation
2025-08-20 13:25:56 +02:00
d70bb634cd
Implement instance tests for timeout
2025-08-17 21:50:16 +02:00
c45fa13206
Initialize last request time on instance start and update timeout handling logic
2025-08-17 21:15:28 +02:00
5e3a28398d
Implement periodic timeout checking for instances
2025-08-17 21:10:48 +02:00
c734bcae4a
Move UpdateLastRequestTime method to timeout.go and add ShouldTimeout method for idle timeout handling
2025-08-17 20:37:20 +02:00
e4e7a82294
Implement last request time tracking for instance management
2025-08-17 19:44:57 +02:00
ccffbca6b2
Add timeout check interval and update instance configuration
2025-08-17 19:26:21 +02:00
902be409d5
Add IdleTimeout option to CreateInstanceOptions and update JSON handling
2025-08-17 19:06:09 +02:00
2abe9c282e
Rename config and instance struct to avoid awkward naming
2025-08-04 19:30:50 +02:00
6a7a9a2d09
Split large package into subpackages
2025-08-04 19:23:56 +02:00
