diff options
| author | Johannes Gäßler <johannesg@5d6.de> | 2024-06-01 08:44:14 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-06-01 08:44:14 +0200 |
| commit | 9b596417af11c9ac44fcae0fcfbc6f3665089083 (patch) | |
| tree | 58ac1a3d1399c597235a9b2247cfa989c2c77665 /examples/server | |
| parent | a323ec60af14a33d560df98f2cc41b4112cb4f80 (diff) | |
CUDA: quantized KV support for FA vec (#7527)
* CUDA: quantized KV support for FA vec
* try CI fix
* fix commented-out kernel variants
* add q8_0 q4_0 tests
* fix nwarps > batch size
* split fattn compile via extern templates
* fix flake8
* fix metal tests
* fix cmake
* make generate_cu_files.py executable
* add autogenerated .cu files
* fix AMD
* error if type_v != FP16 and not flash_attn
* remove obsolete code
Diffstat (limited to 'examples/server')
0 files changed, 0 insertions, 0 deletions