b7668
llama : add use_direct_io flag for model loading (#18166)
Adding --direct-io flag for model loading
Fixing read_raw() calls
Fixing Windows read_raw_at
Changing type off_t to size_t for windows and Renaming functions
disable direct io when mmap is explicitly enabled
Use read_raw_unsafe when upload_backend is available, not functional on some devices with Vulkan and SYCL
Fallback to std::fread in case O_DIRECT fails due to bad address
Windows: remove const keywords and unused functions
Update src/llama-mmap.cpp
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
Co-authored-by: jtischbein jtischbein@gmail.com Co-authored-by: Georgi Gerganov ggerganov@gmail.com
macOS/iOS:
Linux:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: