b7690
server: fix n_cmpl not skipping processing prompt (#18663)
server: fix n_cmpl not skipping processing
fix infinite loop on empty batch
cont : init child samplers + modify child logic
cont : cleanup
cont : improve n_cmpl logic
- launch the parent task first so it finds the slot with best cache
- parent task waits for child tasks to be launched
- when a child task finishes - remove its cache
cont : remove redundant function
cont : reduce parent checks
fix : nullptr task dereference
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
macOS/iOS:
Linux:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: