Support system-installed ggml

[benniekiss]

Is your feature request related to a problem? Please describe.
It would be really helpful if this package was able to find system-installed ggml libraries. Currently, even though I build llama.cpp and this package with CMAKE_ARGS="-DLLAMA_USE_SYSTEM_GGML=ON", the package still fails to find and import the ggml libraries.

In particular, it fails at this line when trying to load the library from the package's root dir:

llama-cpp-python/llama_cpp/_ggml.py

Line 21 in f9f8669

libggml = ctypes_ext.load_shared_library("ggml", libggml_base_path)

Describe the solution you'd like
Better path finding and support for external ggml libraries.

Describe alternatives you've considered
Modifying the code directly.

Additional context

[JamePeng]

I roughly know that some ggml dynamic libraries are installed in the bin folder under the site-packages directory.
But why use this one? LLAMA_USE_SYSTEM_GGML, Python doesn't need this CMake parameter.

[benniekiss]

I need it because I also use whispercpp bindings, and when both provide their own ggml libraries with the wheel, it balloons storage space and quickly leads to conflicts if either one uses a different ggml version.

Llama.cpp and by extension this package needs LLAMA_USE_SYSTEM_GGML in order to not build the internal ggml version.

FWIW, the upstream llama-cpp-python loads the system library without issue, but that is far outdated

[JamePeng]

The original llama-cpp-python code itself doesn't capture system ggml; you just didn't see the error because it didn't use any extension libraries.
In the latest commit, I added searches for lib paths on Linux and macOS systems; you can try it out. Let me know if any paths are missing.

[benniekiss]

you just didn't see the error because it didn't use any extension libraries.

Ah, that makes sense.

I will circle back to testing this soon! Thank you for the quick response and taking up this fork

[JamePeng]

I've upgraded load_shared_library to include a path list feature (allowing you to add custom paths in addition to the default ones). You can later add your own paths to the libggml_base_paths candidate list in _ggml.py, such as those not commonly used Python paths.