summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-09-20examples : fix benchmark-matmult (#1554)Georgi Gerganov
The precision for Q4_0 has degraded since #1508
2023-09-18make : restore build-info.h dependency for several targets (#3205)Cebtenzzre
2023-09-18ci : switch cudatoolkit install on windows to networked (#3236)Erik Scholz
2023-09-17CUDA: fix peer access logic (#3231)Johannes Gäßler
2023-09-17CUDA: enable peer access between devices (#2470)Johannes Gäßler
2023-09-17llama.cpp : show model size and BPW on load (#3223)slaren
2023-09-17CUDA: fix scratch malloced on non-main device (#3220)Johannes Gäßler
2023-09-16Enable BUILD_SHARED_LIBS=ON on all Windows builds (#3215)IsaacDynamo
2023-09-16Enable build with CUDA 11.0 (make) (#3132)Vlad
* CUDA 11.0 fixes * Cleaner CUDA/host flags separation Also renamed GGML_ASSUME into GGML_CUDA_ASSUME
2023-09-16Fixing the last deviations from sentencepiece indicated by test-tokenizer-1 ↵goerch
(#3170) * Fix für #2721 * Reenable tokenizer test for LLaMa * Add `console.cpp` dependency * Fix dependency to `common` * Fixing wrong fix. * Make console usage platform specific Work on compiler warnings. * Adapting makefile * Remove trailing whitespace * Adapting the other parts of the makefile * Fix typo. * Fixing the last deviations from sentencepiece indicated by test-tokenizer-1 * Simplify logic * Add missing change... * Fix ugly compiler warning * llama_tokenize should accept strings containing NUL now * Adding huichen's test case
2023-09-15examples : add compiler version and target to build info (#2998)Cebtenzzre
2023-09-15check C++ code with -Wmissing-declarations (#3184)Cebtenzzre
2023-09-15fix build numbers by setting fetch-depth=0 (#3197)Cebtenzzre
2023-09-15llama : add support for StarCoder model architectures (#3187)Meng Zhang
* add placeholder of starcoder in gguf / llama.cpp * support convert starcoder weights to gguf * convert MQA to MHA * fix ffn_down name * add LLM_ARCH_STARCODER to llama.cpp * set head_count_kv = 1 * load starcoder weight * add max_position_embeddings * set n_positions to max_positioin_embeddings * properly load all starcoder params * fix head count kv * fix comments * fix vram calculation for starcoder * store mqa directly * add input embeddings handling * add TBD * working in cpu, metal buggy * cleanup useless code * metal : fix out-of-bounds access in soft_max kernels * llama : make starcoder graph build more consistent with others * refactor: cleanup comments a bit * add other starcoder models: 3B, 7B, 15B * support-mqa-directly * fix: remove max_position_embeddings, use n_train_ctx * Update llama.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * Update llama.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * Apply suggestions from code review Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * fix: switch to space from tab --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-09-15common : do not use GNU zero-length __VA_ARGS__ extension (#3195)Cebtenzzre
2023-09-15metal : fix bug in soft_max kernels (out-of-bounds access) (#3194)Georgi Gerganov
2023-09-15convert : make ftype optional in simple scripts (#3185)Cebtenzzre
2023-09-15sync : ggml (Metal F32 support + reduce ggml-alloc size) (#3192)Georgi Gerganov
* sync : ggml (Metal F32 support + reduce ggml-alloc size) ggml-ci * llama-bench : fix ggml_cpu_has_metal() duplicate function ggml-ci
2023-09-15cmake : fix building shared libs for clang (rocm) on windows (#3176)Engininja2
2023-09-15flake : use pkg-config instead of pkgconfig (#3188)Evgeny Kurnevsky
pkgconfig is an alias, it got removed from nixpkgs: https://github.com/NixOS/nixpkgs/blob/295a5e1e2bacd6e246db8b2bb35d2a9415883224/pkgs/top-level/aliases.nix#L1408
2023-09-15metal : relax conditions on fast matrix multiplication kernel (#3168)Georgi Gerganov
* metal : relax conditions on fast matrix multiplication kernel * metal : revert the concurrnecy change because it was wrong * llama : remove experimental stuff
2023-09-15cmake : fix llama.h location when built outside of root directory (#3179)Andrei
2023-09-15ci : Cloud-V for RISC-V builds (#3160)Ali Tariq
* Added Cloud-V File * Replaced Makefile with original one --------- Co-authored-by: moiz.hussain <moiz.hussain@10xengineers.ai>
2023-09-15llama : remove mtest (#3177)Roland
* Remove mtest * remove from common/common.h and examples/main/main.cpp
2023-09-14llama : make quantize example up to 2.7x faster (#3115)Cebtenzzre
2023-09-14flake : allow $out/include to already exist (#3175)jneem
2023-09-14cmake : compile ggml-rocm with -fpic when building shared library (#3158)Andrei
2023-09-14flake : include llama.h in nix output (#3159)Asbjørn Olling
2023-09-14make : fix clang++ detection, move some definitions to CPPFLAGS (#3155)Cebtenzzre
* make : fix clang++ detection * make : fix compiler definitions outside of CPPFLAGS
2023-09-14CI: add FreeBSD & simplify CUDA windows (#3053)Alon
* add freebsd to ci * bump actions/checkout to v3 * bump cuda 12.1.0 -> 12.2.0 * bump Jimver/cuda-toolkit version * unify and simplify "Copy and pack Cuda runtime" * install only necessary cuda sub packages
2023-09-14falcon : use stated vocab size (#2914)akawrykow
2023-09-14cmake : add relocatable Llama package (#2960)bandoti
* Keep static libs and headers with install * Add logic to generate Config package * Use proper build info * Add llama as import library * Prefix target with package name * Add example project using CMake package * Update README * Update README * Remove trailing whitespace
2023-09-14docker : add gpu image CI builds (#3103)dylan
Enables the GPU enabled container images to be built and pushed alongside the CPU containers. Co-authored-by: canardleteer <eris.has.a.dad+github@gmail.com>
2023-09-14gguf-py : support identity operation in TensorNameMap (#3095)Kerfuffle
Make try_suffixes keyword param optional.
2023-09-14feature : support Baichuan serial models (#3009)jameswu2014
2023-09-14speculative : add heuristic algorithm (#3006)Leng Yue
* Add heuristic algo for speculative * Constrain minimum n_draft to 2 * speculative : improve heuristic impl * speculative : be more rewarding upon guessing max drafted tokens * speculative : fix typos --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-09-13whisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096)goerch
* Fix für #2721 * Reenable tokenizer test for LLaMa * Add `console.cpp` dependency * Fix dependency to `common` * Fixing wrong fix. * Make console usage platform specific Work on compiler warnings. * Adapting makefile * Remove trailing whitespace * Adapting the other parts of the makefile * Fix typo.
2023-09-13cmake : add a compiler flag check for FP16 format (#3086)Tristan Ross
2023-09-13CUDA: mul_mat_q RDNA2 tunings (#2910)Johannes Gäßler
* CUDA: mul_mat_q RDNA2 tunings * Update ggml-cuda.cu Co-authored-by: Henri Vasserman <henv@hot.ee> --------- Co-authored-by: Henri Vasserman <henv@hot.ee>
2023-09-13speculative: add --n-gpu-layers-draft option (#3063)FK
2023-09-12arm64 support for windows (#3007)Eric Sommerlade
Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>
2023-09-13CUDA: fix LoRAs (#3130)Johannes Gäßler
2023-09-11CUDA: fix mul_mat_q not used for output tensor (#3127)Johannes Gäßler
2023-09-11CUDA: lower GPU latency + fix Windows performance (#3110)Johannes Gäßler
2023-09-11cmake : support build for iOS/tvOS (#3116)Jhen-Jie Hong
* cmake : support build for iOS/tvOS * ci : add iOS/tvOS build into macOS-latest-cmake * ci : split ios/tvos jobs
2023-09-11CUDA: add device number to error messages (#3112)Johannes Gäßler
2023-09-11metal : PP speedup (#3084)Kawrakow
* Minor speed gains for all quantization types * metal: faster kernel_scale via float4 * Various other speedups for "small" kernels * metal: faster soft_max vial float4 * metal: faster diagonal infinity Although, to me it looks like one should simply fuse scale + diagnonal infinity + soft_max on the KQtensor. * Another faster f16 x f32 matrix multiply kernel * Reverting the diag infinity change It does work for PP, but somehow it fails for TG. Need to look more into it. * metal: add back faster diagonal infinity This time more carefully * metal : minor (readibility) --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-09-10convert: remove most of the n_mult usage in convert.py (#3098)Erik Scholz
2023-09-09metal : support for Swift (#3078)kchro3
* Metal support for Swift * update * add a toggle for arm/arm64 * set minimum versions for all platforms * update to use newLibraryWithURL * bump version Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com> --------- Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>
2023-09-09metal : support build for iOS/tvOS (#3089)Jhen-Jie Hong
generated by cgit v1.2.3 (git 2.25.1) at 2025-09-30 23:54:15 +0000