diff options
| author | Radoslav Gerganov <rgerganov@gmail.com> | 2024-05-14 14:27:19 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-14 14:27:19 +0300 |
| commit | 5e31828d3e35c76ecfee665bc23771a4bec1d130 (patch) | |
| tree | 7f5f2edc7c3fc3e7655904316897e32202edd5d6 /examples/pydantic_models_to_grammar.py | |
| parent | 541600201e6480f54ae09e58d16b154d4b4b331d (diff) | |
ggml : add RPC backend (#6829)
* ggml : add RPC backend
The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).
* set TCP_NODELAY
* add CI workflows
* Address review comments
* fix warning
* implement llama_max_devices() for RPC
* Address review comments
* Address review comments
* wrap sockfd into a struct
* implement get_alignment and get_max_size
* add get_device_memory
* fix warning
* win32 support
* add README
* readme : trim trailing whitespace
* Address review comments
* win32 fix
* Address review comments
* fix compile warnings on macos
Diffstat (limited to 'examples/pydantic_models_to_grammar.py')
0 files changed, 0 insertions, 0 deletions